In the following section, issues and future directions of research that were elicited during the VICON project and the VUMS cluster are presented including the derived open questions. These results come mainly from the work in D5.1 especially from the desk research and the designer questionnaires.
1. VICON users’ needs research targeted only to physical disabilities
As described above in the frame of WP01, the presented research of VICON and the majority of similar research projects is targeted towards modelling of persons with physical disabilities and impairments such as visual, hearing and motor. Some of the research efforts are also covering some cognitive disabilities. User models need to cover all aspects of disability to really make a significant difference. Some user models consider visual and motor impairments and do not involve other disabilities as hearing impairments or cognitive impairments. The extension of user models to become complete models covering all disabilities implies a lot of work in user studies and statistical analysis. Discussions in the area of user modelling for accessibility often refer to user needs, requirements and preferences.
However there is still no formal definition of how important and necessary is each level of need. In terms of user interface adaptation, such a definition could help determining the importance and necessity for each possible adaptation and their prioritization.
- Q1: how can a standard for user models be established which covers a wide spectrum of disabilities (physical and mental), user needs and preferences, determining the importance and necessity for each possible adaptation and their prioritization?
2. Privacy concerns
Privacy concerns were not in the forefront of VICON research. The majority of recent models are based on totally different technologies posing issues with regard to interoperability and to their utilisation in different environments and contexts. Privacy as a crucial aspect is not well considered in the majority of existing systems. There is the opinion that there are no privacy issues, if the system uses stereotypes and not specific preferences of the user. The usage of such stereotypes or general data can be used as implicit hints about the specific user preferences or abilities/disabilities. Therefore, privacy in terms of user modelling is one of the very important issues to be tackled especially to help adoption of such systems from wider audiences.
Given the discussions on privacy there is a need for research and development of appropriate mechanisms to address privacy concerns for the following issues: User involvement in managing the privacy of their own profiles. Information exchange mechanisms and rules (technical and possibly legislation). Data gathering and storing techniques and mechanisms.
- Q2: What is the most appropriate mechanism to address privacy concerns for: User involvement in managing the privacy of their own profiles, Information exchange mechanisms and rules (technical and possibly legislation), and for Data gathering and storing techniques and mechanisms?
3. VICON virtual user model based on a mixed approach of user characteristics and preferences
One of the biggest, most interesting discussions in the area of user modelling is the type of modelling to be followed in the area. In general there are two schools of thought.
The first one is basing user modelling on user characteristics. This way a system knowing about the characteristics of a person and other contextual information such as the device used and other environmental variables can reason on what kind of adaptations are needed in order to make the product accessible to the user.
The second approach is basing its modelling approach on user preferences. In this type of systems instead of user characteristics the system keeps information about the user preferences. These preferences could in some cases be chosen by the user herself over a short training/question answering session or could be recorded on the background by keeping track of users’ actions and reactions on various adaptations. When a new user starts using the system is mapped with similar users in terms of choices and preferences and the system is then able to suggest further adaptations.
VICON based its model mainly on the first approach of user characteristics mixing some user preferences trying to benefit of both approaches.
Both approaches have strengths and weaknesses. The first user characteristics based approach is more flexible in terms of taking contextual parameters under account. However, if user characteristics are not described accurately for a person might mean adaptations that will not work for him/her. Therefore, such solutions are quite common in systems aiming for developers and product designers that can describe with such systems their potential users and experiment with techniques such as virtual user models in order to find out accessibility problems on their designs.
On the other hand the second preferences based approach is easier for user to train and can provide easier and more accurate adaptations of interfaces in real-time. However, such systems are usually more difficult to be ported on various devices and software making them quite specific.
The aforementioned observations lead researchers currently to believe that there is a need for more research on how to bridge these two approaches under schemes that could benefit from each one’s strengths and eliminate their weaknesses.
- Q3: How can the strengths of well-established and recognized user modelling approaches be integrated in order to establish a seamless link between user characteristics and user preferences?
Knowing, which groups of people have specific preferences and under what circumstances could lead to systems being able reason based on both user characteristics and preferences. In addition, systems aiming at designers would be better backed up on their suggestions by real user’s feedback and preferences being taken under account. VICON incorporated contextual information in its model
As described above in WP02 and WP03, the VICON user model incorporated as well context and task models. Our experience shows that user interface adaptation can be addressed at design time, run time or both. Utilising user model technology to support designers to create inclusive user interfaces means incorporating it throughout the entire design process. The designer should already be aware of the user requirements already at the sketch phase, when she sketches the interface using paper and pencil. In later stages there should be tools e.g. in a CAD environment to support the designer in creating inclusive designs. In this area we may find two different types of user interface, the pure software interface and / or the hardware interface e.g. of a washing machine or a mobile phone.
The user interface adaptation based on user model technology means that the system knows which user or user type is using the system at runtime and so it can adapt the user interface according to the user’s preferences. However, a very crucial parameter discussed and point out is that user models cannot provide actual benefits to users if there is no connection to context.
A user model describing a user and his/her preferences for a washing machine on the person’s kitchen could be totally different when the same person is using a mobile device his living room. Context can include devices (tablet, mobile phone, TV, PC etc.), environmental conditions (lightning, noise etc.), intent of use / task (work, leisure, education etc.) or even social and cultural background (being on a meeting at work, being alone in my living room, being on my car driving to work or on a holiday trip etc).
Context plays a significant role in reasoning what is the best adaptation to choose for a user under specific circumstances. One of the biggest issues already presented is the connection between user characteristics and preferences. Contextual information makes this connection even more difficult to build. Therefore, although user modelling might focus on user characteristics and user preferences it is necessary to connect with contextual information in order to provide the best solutions for the user.
This means that apart from the standardization and integration efforts that need to be done on user models research on the domain should also be connected with standards and models related with sensor and sensor network technologies. This way the link between user characteristics and preferences will become even easier to build. Therefore, in order to achieve the goal of developing inclusive user interfaces in the majority of cases there is the need for further models rather than the user model e.g. application model, environment model and task model. The mapping between the user characteristics and / or preferences and components of these models may require special knowledge rules of the domain and specialised algorithms.
4. VICON user model is an extensible model
User models are usually not static nature but subject to continual updating e.g. older people are in continuous decline of capabilities. And as long as they use a device, they are going to need to update their profile as capabilities will change, not only on a day-to-day basis, but potentially more often than that, given the equipment that they are using at specific times, moving from one device to another. These continuous updates require the usage of a management system to cope with this aspect.
The aspect of continuous update should be under the control of the user, as it may mean that the user interfaces changes as well continuously. In VICON the users of the system are designers and VICON provided an interface for updating and managing the user profiles. Nevertheless the system of VICON does not include an automatic mechanism for update of the model like machine learning based ones.
Machine learning algorithms are a good vehicle for automatic and continuous user model update as they use the data gathered from the usage behaviour of the user and infer from them the preferences of that specific user or user group. A very useful technology for this process of connections discovery between user models and preferences could be machine learning technology and algorithms. Machine learning technology combined with user models and the abundance of contextual and usage information that today’s devices and systems can provide can help in discovering new connections between users and preferences and also make the training process of such adaptation systems quicker and easier for the end-user. Such components should be considered in future research.
- Q4: How can user characteristics and user preferences be combined and integrated to support the product development process with the help of machine learning algorithms in order to enable a continuous extension of user models?
Likewise the possibility of a “dynamic virtual user model” arises which is capable of combining and integrating user characteristics and preferences.
- Q5: How can the VUM be extended in order to evolve to a “dynamic virtual user model” which is characterised by dynamic properties in order achieve a concise integration of usage and context data provided e.g. through real-time monitoring of interactions with user interfaces and consumer products?
In this manner, a dynamic virtual user model may be interoperable with a diverse range consumer product user interfaces such as washing machines, mobile phones, and TV sets. At the same time, a seamless exchange of user and context data with other sophisticated user models shall be possible, which would finally allow a qualitative and quantitative extension of user profiles.
5. VICON / VUMS – Standards, ontologies and interoperability
VICON as a member of the VUMS cluster has made a lot of efforts towards standardisation and interoperability providing the VICON ontologies, developing converters between the project models and contributing to the VUMS standard. One of the biggest challenges faced today in the user modelling domain is the fragmentation of user model definitions.
Projects employing user modelling techniques for accessibility often produce new definitions of user models depending on their needs, user group focus and targets. This makes quite difficult for models to be used across different projects and research efforts and hinders their progress. Differences in research aims and user group focus justifies to an extent why a new project employing user modelling to provide an accessibility solution would start building its own models. However, now that user modelling appears as a technology used on an increasing number of research and development projects it is crucial to start working on building standards for user modelling in that domain.
Many researchers are actually supporting the idea of having standards for describing users based on models such as the medical model. On the other hand there seems to be an agreement that models describing user preferences are also quite useful and need to be worked on towards standardization. Given the discussions about the user characteristics based (medical) models and the user preferences based models and the emerging need for both of them, it is obvious that both approaches should also be supported by standardization actions. This way the emerging need for research on the connections between them will be easier to fulfill.
The fragmentation of solutions and models in the area for user modelling for accessibility hinders evolution of research and development because it is making more difficult for research and development teams to cooperate and exchange information, knowledge and build on each other’s work.
Although there are already actions for standardization of user models there is not yet enough support and a critical mass of researchers and practitioners behind them to support them. Therefore there is a need for encouragement and increasing the support on such standards in order to increase adoption and further improve and extend them.
- Q6: How can a user model standard such as developed within the VUMS cluster be further extended and matured in order to establish a critical mass of supporters and stimulate adoption?
6. VICON is based upon manual Adaptation in the CAD phase
In the VICON software framework the CAD tool (integrated into Siemens NX) requires a manual annotation of interaction components to maintain the functionality of an application of recommendations to a currently defined 3D product model. To support the designer and with respect to the increasing number of components, the model presents possible UI components to the designer.
In order to realise at least a semi-automatic adaptation of the model, the VUM could be substantially extended by applying machine learning algorithms which implicit the functionality to provide a semi-automatic support of the annotation process.
- Q7: Regarding the realisation of a semi-automatic support in the CAD phase, it is essential to determine how a semi-automatic segmentation and a recommendation of CAD parameter values to an existing virtual product design can be achieved?
7. VICON Focuses upon static hardware UIs
The focus of VICON was upon static, non-adaptive user interfaces. That means the characteristics of the UI were based upon onetime captured requirements manifested within the VUM. However, in smart environments which are characterised by the presence of technology-enhanced objects and the fact that users become more versatile, it would be desirable to realise adaptive and personalised hardware user interfaces, which are able to adapt to the real-time requirements of their beneficiaries on the fly. Still, real-time context information is rarely taken into account for the adaption and personalisation of user interfaces. In this manner, adaptation and personalisation may be subject to an ad-hoc customisation to specific interaction modalities or to a dynamic allocation or reconfiguration of functionalities of hardware resources.
A special challenge lies in identifying new approaches in acquiring, processing, and applying contextual data and real-time user needs of diverse user groups while they interact with consumer products in smart environments. Appropriate techniques have to guarantee that the real-time user needs are properly allocated and applied to the respective user interfaces.
- Q8: How must Adaptive Hardware User Interfaces be designed for user groups with age-related physical impairments in order to provide the most appropriate support/assistance in interacting with smart environments?
For this purpose a sophisticated design support framework will be necessary, suitable for designing adaptive hardware user interfaces with characteristics such as personalisation, transferable user profiles and autonomous context recognition. In this respect it is legitimate to consider the capability of the VICON toolset in order to fulfil this requirement.
- Q9: How can the model-based framework of VICON be extended in order to enable device makers and user interface designers to create and deploy networked smart interfaces automated, which contain context- and usage-based interface adaptation mechanisms to address personalisation and accessibility needs of a diverse user base?
8. VICON provided an effort to standardising hardware design patterns
In the final project phase, the need emerged to improve the textual design recommendations through a better presentation and visualisation. The benefit of standardized design patterns is obvious as it overcomes the challenge of presenting and communicating information to the designers in a usable and easily adaptable manner. In VICON this was initially achieved through transferring each of the 75 textual design recommendations into a structure, based upon HCI design patterns. The approach of HCI design patterns is not new, but unique is the fact that it was applied upon hardware user interfaces since the majority of currently existing HCI design patterns usually refer to software user interfaces.
- Q10: What are the essential characteristics of a design pattern language for physical user interfaces?
Complimentarily it should be ensured that design knowledge is constantly held up-to-date. The efforts are large to maintain and improve virtual user models and design patterns. Therefore an evolving knowledge base would be desirable where designers are engaged to contribute with their own inclusive design experience which flows back into the virtual user model. In this way, it is secured that the virtual user model evolves continuously to the benefit of the designers.