Critical Examination of Project Findings

Overview of the Project Findings

The key project findings in relation to the objectives of the project and global lessons learned are highlighted in the table below:

Project Findings of VICON Related Project Objective Description of Project Finding
PF1 OBJ 5

There is a business case for Inclusive Design.

Learned Lesson #1

PF2 OBJ 3

VUMs can contribute to the business case for Inclusive Design.

Learned Lesson #2
Learned Lesson #3

PF3 OBJ 3

VICON is beneficial for designers but VUMs must be designed and integrated carefully.

Learned Lesson #4
Learned Lesson #5

PF4 OBJ 3

Usability of inclusive design tools will always be a key issue which stimulates acceptance.

Learned Lesson #6
Learned Lesson #7

PF5 OBJ 1
OBJ 2
OBJ 4

There are technical limitations of the VUM concept, but these can be overcome.

Learned Lesson #8
Learned Lesson #9

Critical Examination of the Key Project Findings and Lessons Learnt

PF1: There is a business case for Inclusive Design

The beneficiaries’ study of D5.3 has found that usability and ease of use are important factors influencing product purchases by the VICON beneficiaries and that they become more important as the user ages. This is a significant finding as it informs designers that an inclusive design approach can lead to digital products that are not only ‘better’ for people over 65 years of age who have mild to moderate hearing, vision or manual dexterity impairment; but are more likely to be purchased and used by them. 

PF2: VUMs can contribute to the business case for Inclusive Design

The designers study of D5.3 has also enabled us to more clearly state the drivers and barriers to the use of VUMs in product design, such as reduced lead time to market (a driver) and disruption of workflow (a barrier). What is most interesting is the finding that the two biggest advantages of using VUMs – time and other cost savings – are also the two biggest barriers to the adoption of Inclusive Design. This supports the claim that VUMs can be of great benefit for Inclusive Design by increasing its ability to provide a positive return on investment. VUMs should allow companies to develop more inclusive and therefore more financially successful products for the VICON beneficiaries, without sacrificing the need to get time-critical products, such as mobile phones, into the marketplace.

This and the D4.3 study did not allow us to pin point exactly where the designers made key design changes as a result of the VICON toolset. It was therefore difficult to know for certain if a design change was a direct result of the information provided through the VICON toolset or a design decision made independently. When developing a later version, it would be worthwhile to observe the designers in real time (remote testing can be conducted using usability testing software), to identify precisely where the toolset directly led to changes in user interface design. The success of the text recommendations, the virtual simulations and the different applications of the VICON toolset (Sketch, CAD and Virtex) could therefore be more effectively evaluated

If the use of VUM can reduce the time and cost associated with involving real users in testing then it may help to overcome the biggest barriers to inclusive design. In the future an online repository of virtual user models based on the VUMS cluster specifications could be implemented and provided via different adaptive API’s for various applications utilising distinct granularity levels and implementing many use cases e.g. adaptive user interfaces or simulation of human body and abilities. VUMs may reduce costs directly by using free virtual users in place of real users who would have to be paid to take part in development.

While it doesn’t mean that VUMs will replace real user testing altogether, they should allow testing at times where testing with real users is not possible or feasible. VUMs therefore have the potential to reduce the overall product development time and costs required for inclusive design by allowing virtual testing to be carried out earlier and at more frequent intervals than would be possible for real user tests.

Based on the findings of the desk research and the designer interviews, if a VUM toolset is designed to work on designs at the earliest stages and to include user characteristics that are difficult to source or replicate in the user population, it has definite potential to make inclusive design more realisable. This should allow companies to develop inclusive designs without sacrificing the speed of getting time-critical products into the marketplace.

While there is a risk of VUM replacing real user testing entirely, at least for some companies, there are additional potential advantages which may be offset against that risk.

PF3: VICON is beneficial for designers but VUMs must be designed and integrated carefully

Virtual user modelling is relatively new in product development and the VICON virtual user concept adds another layer to this process. Feedback from designers in the evaluation work of D4.3 has shown that the design recommendations produced by VICON are perceived as useful for developing inclusive designs. Areas where further improvement can be made, such as more structured organization and grouping of VICON profiles, have also been revealed.

The focus group work described in D4.4 shows that if a recommendation is presented to a designer at an inappropriate time, the effectiveness of the message and the opportunity to educate the designer at the appropriate point might be missed. For design recommendations to be appropriate, they must take into account the specific product type. For example, the key spacing on a washing machine is less constrained than on a mobile phone where it determines the overall size of a size-critical product. These issues indicate the need to improve the design recommendations part of the tool.

Another finding is that a rule-based VUM would be improved if its rules could take into account interactions between cognition, sensory and physical requirements which suggest conflicting design solutions. For example, where using larger labels may make each label easier to read on its own but lead to more crowding, both reducing readability and increasing cognitive load. Instead of suggesting that the answer to most visual issues is merely to “make labels bigger” the VICON tool should encourage designers to come up with more creative solutions to improve the overall user experience. Design recommendations should really allow designers to come up with new solutions and not just lead to only slight variations of existing solutions.

PF4: Usability of inclusive design tools will always be a key issue which determines acceptance.

For the overall VICON system a number of usability issues were identified. Comments regarding the look and feel of the toolset highlight the fact that designers represent not only a very sensitive group of professionals, but also a user group that has learned certain visual language from specific software that they use every day for their professional needs, e.g. Adobe Photoshop or Siemens NX. Although the VICON CAD Design application uses the standard GUI elements of the NX environment, other VICON tools utilize their own interactive elements and thus, they look and feel different. The feedback of designers suggests that it is indispensable to hire graphic and interaction designers that will intentionally create the overall appearance of the software intended to be used by designers. There are several concrete suggestions for improvement that have been proposed by the designers both for the entire virtual user concept and for the three prototypic applications. Although the VICON design recommendations were overall perceived as useful for realising inclusive designs, further improvements of the VICON toolset can be achieved when the VICON profiles of the Virtual User Model would be organized and grouped in a more structured way. Another identified subject for improvement is that it took relatively long for the designers when going through the design recommendations list. For revising this matter, the participants suggested a more adequate approach for grouping or sorting the recommendations based on their properties, such as importance, in order to achieve a higher degree of comprehension and usability. Besides some minor usability and look and feel issues, the evaluation application Virtex was generally well accepted. The majority of the designers found the textual evaluation report clear structured, understandable and informative. The references for further reading were appreciated. Some designers however criticised the virtual usage simulation and requested a more “scientific” visualisation, which would depict usage difficulties with exactly the same precision as the VUM provides. These designers explained their opinion with the example that the current virtual environment visualisation composes furniture, which is actually not part of the VUM and thus distracts the attention from the main data really used in the simulation. On the contrary other designers liked the concept of the 3d environment. It would therefore be feasible to rethink the concept of the product usage visualisation in future projects.

It could be confirmed that when implementing inclusive designs, the VICON toolset provided an appropriate spectrum of information for most of the designers and product engineers who participated in the VICON evaluation. However it is important to note that VICON focussed only on hearing, vision and manual dexterity, so the information provided to designers only addressed inclusive design issues under those categories. Cognitive issues and the complex interaction between cognitive, sensory and physical inclusive design issues were not addressed.

One of the most important conclusions is that the VICON toolset integrates smoothly within the usual design workflow and is not cumbersome for designers in their work. A clear majority of the survey participants responded that the internal workflow of VICON toolset was clear and straightforward. However opinions on the idea of integrating the evaluation application Virtex, which is currently standalone, into the CAD design environment were divided. Since some designers don’t work with CAD tools, they expect a standalone product evaluation tool. But some designers desire the product evaluation section to be integrated within their usual work environment, namely CAD, because they prefer immediate feedback to their product design modifications.

PF5: There are technical limitations and barriers of the VUM concept, but these can be overcome

The user and virtual evaluations of D5.1 showed that the implementation of virtual testing is currently very tedious, requiring intensive effort. More work is needed to create more realistic representations of virtual users, their movements and environments, based on the VUMS cluster specifications. Many details e.g. emotional state and attention of the real users have gone lost during the creation of the abstract models.

One main key finding based on the beneficiary trials (D4.4) is the non existence of information about possibilities of usage of a product. For instance regarding the usage of a mobile phone, twelve different ways of holding and using a handheld product were reported:

  • One-handed use in right hand, thumb to press buttons
  • One-handed use in left hand, thumb to press buttons
  • One handed use, product sitting on table, right hand to press buttons
  • Two-handed use, hold in left hand, right forefinger to press buttons
  • Two-handed use, hold in left hand, right middle finger to press buttons
  • Two-handed use, hold in left hand, right thumb to press buttons
  • Two-handed use, hold in right hand, left forefinger to press buttons
  • Two-handed use, hold in right hand, left thumb to press buttons
  • Two-handed use, hold in right hand, use left forefinger and thumb to press buttons
  • Two-handed use, hold in both hands, both thumbs to press buttons
  • Two-handed use, hold in both hands, right thumb to press buttons
  • Two-handed use, hold in both hands, left thumb to press buttons

Regarding impairments, this issue is even more important. Before performing a task, impaired customers think about possible ways how (and in which position) this task execution is as convenient as possible. Especially manual dexterity impaired beneficiaries do have a need to execute a task in another way as “standard” execution can be not possible.

Designers mainly focus on one task execution method of how to perform a task using a device. The VUM and especially the task model does not include different kinds of task execution methods. Regarding further implementations, an additional task execution model between the task and recommendation model based on user trials to define options how to perform a single task is advantageous. This model should include parameters and values regarding the complete process of task execution.

For instance during user trials with beneficiaries in D4.4, one recommendation regarding washing machine panels was to place the input panel diagonally regarding the possibility to read in a standing position. In the final release of the VUM, the task model is dependant on the current device and included during reasoning.

The final VUM as presented in D2.2 can be enhanced by a task execution model by addition into the ontology reasoning steps between the connection of tasks and recommendations. By using this, one task resulting in a variety of task execution methods results in a disjunct set of recommendations based on each execution method.

Finally, the availability of analysis algorithms and recommendation for user interfaces of consumer products is still in its infancy. In this stage there is still a lot of effort required in order to create a corpus of such algorithms and recommendations. Further work will be necessary regarding the enhancement of the graphical presentation of the virtual users and their movements to be nearly realistic like real users and environments.