Pier Luigi Emiliani
National Research Council of Italy
Institute of Applied Physics “Nello Carrara”
+39 055 5226452, firstname.lastname@example.org
The objective of the CARDIAC (Coordination Action on R&D in Accessible and Assistive ICT) project is to produce prioritized research roadmaps in selected aspects of the application of ICT in the eInclusion of people with activity limitations, using the SDDP (Structured Dialogue Design Process) methodology. This allows a free discussion on topics of interest, a democratic selection through voting of possible actions to be carried out in order to reach a set objective, and the identification of priorities in carrying out them.
One of the important aspects of the emerging information society is the importance of network services that must be accessible and supportive in order to favour inclusion of citizens. The network and its services can be considered at least at four different levels: (1) as a repository of information (the Web – mainly the present situation); (2) as repository of a basic support to people with activity limitations (see the “Raising the floor” initiative); (3) as a socialising virtual environment; (4) as an explicit support to people.
The Cardiac project is mainly concerned with the last two levels in the previous list. The following aspects are supposed to be considered: (i) ambient intelligence; (ii) mobility and ubiquity; (iii) interoperability, (iv) mainstreaming and design-for-all; (v) possible impact of the use of open-source methods; (vi) relevance in different application environments.
The present paper summarizes the information that, according to the methodology, has been made available to participants in the SDDP on network services and applications in order to specify the topic under discussion and the aspects to be considered. For example, the fact that accessibility of systems and services was the concern of another SDDP meeting and, therefore, the impact of services in explicitly supporting people, when necessary, had to be considered.
eInclusion and Design for All – Mainstreaming eInclusion
Traditionally, the main efforts in considering ICT services and applications from the eInclusion perspective have been in making them available, i.e. “accessible to all potential users”, for example by modifying their interface. More recently, at the political level, eInclusion has been advocated, as a form of Universal Access. For example the Riga Ministerial Declaration (2006) defines eInclusion as the right of people to be granted availability of all information and communication facilities in the Information Society and to be supported by ICT for achieving their goals in all environments. For ICT services and applications this implies (in addition to accessibility), an increased emphasis on contents, whose representation must be suitable for the different users, usability, usefulness (meeting real needs of users) and the ability to support explicitly users. The Riga Declaration also states that “e-Inclusion" means both inclusive ICT and the use of ICT to achieve wider inclusion objectives. Therefore, ICT must be inclusive (i.e. mainly accessible), but also able to explicitly support users to reach eInclusion objectives.
From the methodological perspective, the Universal Access and Design for All approaches toward eInclusion, are explicitly mentioned by the EU Commission (e-Inclusion Ministerial Conference held in Lisbon in 2000) and by the United Nations in the “Convention on the Rights of Persons with Disabilities” (2006), where in the Article 2 Universal Design (the international locution for the European Design for All) is defined as “the design of products, environments, programmes and services to be usable by all people, to the greatest extent possible, without the need for adaptation or specialized design and it is stated that “Universal design shall not exclude assistive devices for particular groups of persons with disabilities where this is needed”.
The DfA approach (Emiliani & Stephanidis 2005,Emiliani, 2009), as compared with the Assistive Technology approach, is often criticized on the basis of the concern that “many ideas that are supposed to be good for everybody aren’t good for anybody”. However, the Design for All methodology in the context of Information Society Technologies is not to be conceived as an effort to advance a single solution for everybody, but as a user centred approach to providing products that can automatically address the possible range of human abilities, skills, requirements and preferences, using the “intelligence” available in them. Consequently, the outcome of the design process is not intended to be a “singular” design, but a design space populated with appropriate alternatives (intelligent adaptation and personalisation).
Finally it is necessary to devise a technical approach, according to which this implementation strategy can become a reality. As shown in several projects funded by the European Commission, a working technical approach is the one based on adaptability and adaptivity (Stephanidis Ed. 2001). This is based on the fact that any system and service should be intelligent enough to be able to automatically adapt its functionalities and interface to each single user, according to her known characteristics or an assumed stereotype (different privacy levels), when she starts to interact and modify them in real time as a function of the usage and context (if allowed to monitor the user behaviour).
Ambient intelligence – The Information Society
In addition to the social component (definition of eInclusion) and the methodological component (Design for All), fundamental changes in considering the ways of dealing with problems of inclusion of people with activity limitations are also due to the present technological developments, which are leading to a new organisation of the society. From a technological perspective, it is foreseen that the society will evolve from an industrial toward an information society. According to many scenarios of development the information society will not be based on an increased use of computers and terminals as presently available, but an ‘Ambient Intelligence” (AmI) environment will emerge populated by a multitude of intelligent devices, no longer perceived as computers, but rather as augmented elements of the physical environment offering useful functionalities (Aarts & Marzano 2003; ISTAG 2003).
Some of the foreseen devices will be personal - hand-held and wearable (e.g., wrist-watches, bracelets, personal mobile displays and notification systems, health monitors), other will be public, available in the surrounding environment (e.g., wall-mounted displays). Computational power and interaction peripherals will be distributed and equipped with facilities for multimodal interaction and alternative input/output, including, e.g., voice recognition and synthesis, pen-based pointing devices, vibration alerting, touch screens, input prediction and with accessories facilitating alternative ways of use (e.g., hands-free kits).
From the user perspective there will be a change of paradigm, from devices that are accessible or support people to services and applications that offer useful functionalities and integrated support to users: artefacts in the environment are interconnected and integrated in an “intelligent” control system, which is able to support their cooperation for favouring inclusion. Individual interactions will be mainly substituted by social networking. Emphasis will not be on technology itself, but on functionalities that the environments could or should offer, irrespective of the technical implementation. This vision is consistent with the WHO approach, according to which difficulties in carrying out activities are not only due to limitations of people but also to contextual factors. Therefore, they can be alleviated not only through accessibility or augmenting the capacities of people, but also making available functionalities which reduce the needed level of capacity.
Even if intelligent objects and multimedia interaction systems are starting to appear (for example, in cars to favour communication and access to information of drivers who are functionally unable to see the contents of a screen and to manipulate a mouse and in smart homes), the real winning factor seems to be the “intelligence” in the environment and in services. It is interesting to figure out what level of intelligence is supposed to be available in the AmI environment. This is immediately clear, if the high level specifications of an AmI environment are considered. The AmI environment should be (ISTAG 2003): (i) Unobtrusive (i.e. many distributed devices are embedded in the environment, and do not intrude into our consciousness unless we need them); (ii) Personalisable (i.e. it can recognize the user, and its behaviour can be tailored to the user’s needs); (iii) Adaptable (i.e. its behaviour can change in response to a person’s actions and environment); (iv) Anticipatory (i.e. it anticipates a person’s desires and environment as much as possible without the need for mediation). It is clear from the list, and particularly from points (iii) and (iv), that intelligence is not meant as having some computer power in the objects and communication among them, but as the ability to “reason” about what people could need in the different living environments.
The nature of interaction in AmI environments will change, evolving from human-computer interaction to human-environment interaction (Streitz 2007). It will shift from an explicit paradigm, in which the attention is on computing tasks made available by the used application, towards an implicit paradigm, in which interfaces themselves drive human attention when required and will take care themselves of individual tasks.
Today interaction is based on pre-programmed solutions for the design space of the systems, while in AmI the interaction space is ill-defined and unpredictable. Therefore interaction may pose different perceptual and cognitive demands on humans compared to currently available technology. One of the main challenges in this respect is to identify and avoid forms of interaction which may lead to negative consequences such as confusion, cognitive overload, frustration, etc.
The new ambient user interfaces (software frameworks for developing and orchestrating ambient interactions) will not probably be instantiated on a computer screen, but they will be: (i) Multimodal and distributed in space (e.g., employ the TV screen and stereo speakers to provide output, and get input through both speech and gestures); (ii) Able to creatively combine the available, dispersed computing devices in order to provide useful, added-value, services; (iii) Tailored to the current needs and characteristics of a particular user and context of use.
The development of networks and services made available on them are one of the most evident components of the emerging information society. Internet and the Web, after their success as a repository of indexed information (if accessible - guidelines for accessibility exist - useful as such for making available a lot of relevant information to users) are developing in many directions of potential interest for favouring eInclusion.
Web 2.0 – Socialisation and services
Recently, the Web has made available a virtual interaction space (Web2.0, Online Collaborative Systems, Social Networks) giving people the possibility of social interaction. Users of the Web have become prosumers, i.e. producers of information to be made available on the Web. Discussion about and modification of the information is possible. From the perspective of people with activity limitations, this can be very important, for example, allowing interactions where lack of abilities of people can be hidden, but it can also have a negative impact on accessibility for the impossibility of obtaining the knowledge and use of accessibility guidelines by non-professionals producers of information.
On the other side these developments can offer additional support opportunities in different living situations.
This is due to the fact that Web 2.0 is also emerging as a Web of services. The services may be made available by service providers (as e.g. Cloud computing), but they can be also created by end-users, with the facilities made available by the Web 2.0 architecture and tools. These services start to have features able to allow adaptation, customisation and control according to the needs of the users. Moreover, they can be composed by the users, for example by mixing existing services and data to implement services which are truly useful for them, matching their needs for any time and in any context of use.
Semantic Web – Intelligence in information
In the original Tim Berners-Lee’s article (Berners-Lee 2001), the term Semantic Web described the evolution from a Web containing mainly documents for humans to read to one that included data and information for computers to manipulate. The Semantic Web activities at W3C provide web sites with standards allowing publishing internet contents in a form that machines can process and integrate more readily. The intent is to enhance the usefulness of the Web and its interconnection capabilities through a common standard (RDF) for websites to expose information "marked up" with semantic information, and then having automated agents to perform tasks for users of the semantic web using this data.
This can be very important for eInclusion from two perspectives. The first is that in many cases accessibility is favoured by the transduction of information from one medium to another. This is made easier by the availability of a description of the information available. Moreover, this is an important component toward the emergence of a really intelligent environment.
Internet of things – Toward ambient intelligence
So far, most people, including e.g. developers in the “Ambient assisted Living” environment, think that the Internet of Things is only having around interconnected sensors through which the health care situation or the position in the house of people can be sensed or just putting Radio Frequency Identification tags on some dull thing so smart people know where that dull thing is. Instead, it is about embedding intelligence so things become smarter and do more than they were supposed, for example supporting people. The Internet of things means the fusion of the physical and digital worlds: (i) Physical entities have digital counterpart; (ii) Objects become context-aware – they can sense, communicate and interact; (iii) Immediate responses can be given to physical phenomena; (iv) Instant information can be collected about physical entities.
Intelligent real-time decision making becomes possible, thus opening up new opportunities to handle incidents, meet business requirements, create new services based on real-time physical world data, gain insights into complex processes and relationships, address environmental degradation (pollution, disaster, global warming), monitor human activities (work, criminal, health, military), improve infrastructure integrity (civil, energy, water, transport), and so on.
This will start to happen when developers of Ambient Intelligence applications will stop dealing only with the easy part of the problem (i.e. the ambient part) and will start dealing with its complex part (i.e. the intelligence component), which does not mean to have a computer on board of any object, but to use it to understand as far as possible what people need.
Life of people in an AmI environment is supposed be facilitated by the availability of different services offering useful functionalities. So far, the inclusion of people with activity limitations has been based on some complementary approaches: adaptation of systems addressing the needs of individual user groups (e.g., by adapting their human computer interfaces), adaptation of services of general use (e.g., alarm services), and creation of special services (e.g., relay services). According to the available development scenarios, it seems that the ambient intelligence environment will made available for all many services up to now mainly used to support people with activity limitations, as the ones briefly described below.
Environmental control systems
Environmental control systems, introduced for the independent living of persons with motor disabilities, become an integral part of the living environment. In principle they can be designed in such a way as to be extendable to incorporate additional facilities (either for general purposes, e.g., robotic systems, or for specialised support, e.g., assistive technologies).
Relay services, of interest for people who cannot hear/and or speak are in principle available by default in the Ambient Intelligence environment, where voice recognition and synthesis, automatic translation, gesture recognition (sign language and lip reading) and animation (synthetic sign language and lips movements) are available. Transduction of information between media (e.g. text to voice, voice to texts, etc.) and automatic subtitling are given for granted.
Navigation systems and services are present or can be used in many scenarios for different purposes. They are supposed to acquire positioning information and to have the possibility of controlling the presence of unpredictable obstacles (people, baggage, etc.), obtainable through the use of features of the AmI itself (e.g., a control system able to monitor tagged objects and communicate with people). If the user has cognitive limitations, the navigation system may tune the level of support to the known abilities or to the perceived present difficulties.
Alarm and support/control services
The entire AmI is a pervasive and very sophisticated alarm and support/control system. This may be very important for example for people with cognitive problems. AmI can continuously control a user’s behaviour in the various environments according to known habits and intervene if necessary, for example reminding of tasks and helping to perform them. When necessary, AmI can also contact the family or a carer for advice and help. If necessary, AmI is able to describe its layout and functionalities, as well as the functionalities of its devices (e.g., the remote control of the hotel room). Moreover, AmI is able to organize sequentially the flow of information and the performance of the necessary tasks, allocating the necessary time.
Broadband communication facilities
The additional opportunities offered by AmI are related to the availability of broadband communication facilities. The possible scenario of a user travelling abroad offers an exemplification of advanced telecommunication facilities, in the car, in the hotel room and in the presentation room. When driving, the user is tracked by the navigation system and people know (if she wants) that she can be contacted. In the hotel room there is an audio/video system, the video scenes of which are described if she cannot see, and automatically captioned, if she cannot hear. The audio/video system can be used also for communication with her daughter, with whom she can not only communicate, but also go through the news as they watch them at the same time from different environments. Obviously, if she cannot see, she listens to the news, while if she cannot hear she can read the news, which is automatically captioned.
Audio/video interpersonal communication services
Even if support by technology can be invaluable in some circumstances, support by other people can be more efficient and acceptable in some situations and activities. It can introduce a personal dimension, which increases acceptability and efficiency in the intervention. AmI, with its emphasis on cooperative activities, whereby people can remotely carry out common activities with audio and visual contact, can increase the feasibility of the approach.
As already mentioned, the real winning factor is the intelligence in the environment. Intelligent agents seem interesting to offer focused services to people. However, they normally address a single “intelligent” task and try to mimic the behaviour of the owner. The problem is much more difficult when the entire environment or complex parts of it must be controlled in a way to show behaviour that people can consider “intelligent”.
At the level of interaction, a smart environment may provide an extremely large number of complex choices. An interface that directly offers all the possibilities to the user may result in it being cumbersome and complex. On the contrary, the user interface should act as an intelligent intermediary between the system and the user. This is the reason why Artificial Intelligence methods and techniques are starting to be used for the development of adaptive intelligent interfaces. Intelligent interfaces are first supposed to be able to adapt to the user’s physical, sensorial and cognitive capabilities, some of which may be restricted due to aging or impairments and/or may change along the day, due to e.g. fatigue, and changes in motivation. Then, an important characteristic of the human interfaces for smart environments is their spatial dependency. Many features and possible effects of interaction depend on the position of the user. For instance, a simple command as "switch on the lights" must be differently interpreted according to the place where it has been given. Provided that the user is located with enough precision, the interface needs a spatial model to be able to decide what the lights to be switched on are. In addition, the interface, in order to avoid potential dangers, must be able to decide the services that can be offered to the user in the current location. In contrast to graphical user interfaces, ambient user interfaces should take advantage of the available AmI Infrastructure, in order to support interaction that is tailored to the current needs and characteristics of a particular user and context of use.
Moreover, the AmI environment must take care of the contexts of use. In AmI, the situation is very complex, because in the ubiquitous interaction with information and telecommunication systems the context of use may change continuously or abruptly and the same systems or services may need to behave differently in different contexts. It is not sufficient for a system or service to behave correctly at a given instant, but it must continue to behave correctly for the entire process. This requirement is coherent with the idea that in AmI intelligence must be essentially in the environment and not in the individual objects. Functionalities will be surely more easily available if they can migrate through the network, instead of being entrapped in objects.
Another important issue is avoiding the risk of a possible mismatch between the model of interaction of the system and the user’s mental model of it. As a matter of fact it must be considered that in today systems, designers have pre-programmed solutions for the design space of the systems, while in AmI the interaction space is ill defined and unpredictable. In a context, defined as above, three level of abstractions exist: the sensing level (numeric observables), the perception level (symbolic observables), and the level of the identification of situation and context. This is the level where the conditions for moving between situations are identified and needs of the user and of the system are anticipated. This requires replacing explicitly coded responses to situations and contexts, with a higher-level, more knowledge-intensive use of machine-readable strategies coupled with reasoning and learning.
In AmI the emphasis is on (abstract) goals of the users that the environment must infer and structure in a set of tasks adapted to the users themselves and the context of use (for example without interfering with the goals of other persons in the same environment). The acceptability and uptake of the new paradigm will be essentially dependent on how smart the system is in inferring the goals (desires) of the users in the varying contexts of use and in organising the available resources (intelligent objects, services and applications in the environment) in order to help users to fulfil them. This means that an “intelligent” control must be available. So far most of the available control systems are deterministic. This is not compatible with the emerging situation for two main reasons. The first is that AmI is not only concerned with measurements from sensors, but with goals of people to be fulfilled and interaction in a social context. Moreover, it must take into account that the emerging model may be a social group interacting in order to cooperate for carrying out activities connected with independent living and interactions within a social environment.
Finally, ambient intelligence is also supposed to inspire trust and confidence and to be controllable by ordinary people. The requirement about trust and confidence is very challenging, because they can be obtained only if the user has a complete knowledge at the conceptual level of the running principles of the systems, services and applications and is given the possibility of controlling all the steps necessary to obtain the required results. For what concerns control by ordinary people, sometimes, a simple and naïve concept is assumed, i.e. that the user is given the possibility of switching off the system, service or application. But the problem is not so simple. For example, switching off the telephone can be a problem if a user is connected through it to an alarm system or a health care monitoring system. Therefore, it is necessary that the AmI environment is able to cooperate with the users, according to their profile (e.g. culture, technical knowledge, and possible impairments), the context of use, and the emotional situation in order to find a compromise between privacy or fatigue etc., and possible security aspects.
The technological trends seem well established as shortly summarised in the present document. From the perspective of people with activity limitations, it seems that most of the technology of interest for their inclusion (for example the first application of voice synthesis was in screen readers for blind people) will now become of interest for products for the general public. Moreover, services that until now have been set up to support people with activity limitations (for example environmental control systems, remote control, navigation) are foreseen as part of the very fabric of the Information society.
However, there is always a concern at the back of the mind of people working in eInclusion. There will be a really positive impact, only if the technology and services to be developed take explicitly into account all people who potentially need them. Thus, the purpose of the SDDP on network services is to clarify the meaning and importance of this “if” and to propose activities necessary, if any, for favouring the emergence of an Information Society where telecommunication services are really instrumental in granting equal access to all opportunities, by all. Additional information about technological developments, impact on eInclusion and necessary research activities can also be found in (Emiliani et al., 2008; Emiliani et al, 2009, Roe Ed, 2006).
It is interesting to observe that concerns about a possible use of technology without sufficiently taking into account the real needs of people have also been expressed at the European level. In one of the last ISTAG documents (ISTAG, 2011), the experts who have developed the vision of Ambient Intelligence write:
“A balanced progress perspective is needed here: the future requires a continuous balancing of often contradictory individual, social, societal, entrepreneurial, and ecological needs. We cannot simply assume that technical developments in ICTs will lead to socio-economic progress. We need to ask ourselves why and when we need more transistors, faster clock speeds, and more bits. More than ever we need to put human beings in their social environment in the centre. We need to articulate and communicate what social objectives publicly funded innovation aims at. This requires both the involvement of new stakeholders and a stronger focus on social innovation. Social innovation refers to new strategies, concepts, ideas and organisations that meet social and societal needs. In our view technological innovation and social innovation are inextricably linked and inter-connected in the near future, as the eInfrastructure is growing out to be a basic and vital societal infrastructure. This mutual shaping process implies an understanding of ICTs as a transformative force that redefines both problems and solutions. This not only involves a new approach towards R&D&I, but also requires new indicators to measure productivity, growth, development, and progress.”
Aarts, E. & Marzano, S., (2003). “The New Everyday views on Ambient Intelligence”. Rotterdam: OIO Publishers.
Berners-Lee, T., Hendler, J. & Lassila, O. (2001). 'The Semantic Web', Scientific American, 34-43.
Emiliani, P. L. & Stephanidis, C. (2005), “Universal access to ambient intelligence environments: opportunities and challenges for people with disabilities”, IBM Systems Journal 44(3), 605--619.
Emiliani, P., L., Burzagli, L., Billi, M., Gabbanini, F., Palchetti, E. (2008). Report on the impact of technological developments on eAccessibility. DfA@eInclusion Deliverable D2.1. available at: http://www.dfaei.org/deliverables/D2.1.pdf.
Emiliani P.L (2009). Perspectives in Accessibility: From Assistive Technology to Universal Access and Design for All, in “The Universal Access Handbook”, C. Stephanidis (Ed.), CRC Press - Taylor and Francis Group, pp. 2.1 – 2.18.
Emiliani, P., L., Aalykke, S., Antona, M., Burzagli, L., Gabbanini, F., Klironomos, I. (2009). Document on necessary research activities related to DfA. DfA@eInclusion Deliverable D2.6. Available at: http://www.dfaei.org/deliverables/D2.6.pdf.
ISTAG (2003). “Ambient Intelligence: from vision to reality. For participation in society and business”, Information Society Technologies Programme of the European Union Commission (IST).
ISTAG (2011). Orientations for EU ICT R&D & Innovation beyond 2013.
Roe, P. (ed.) (2006). “Towards an Inclusive Future: Impact and Wider Potential of Information and Communication Technologies”, COST, Brussels.
Stephanidis, C. (ed.) (2001), “User Interfaces for All – Concepts, Methods and Tools”, Mahwah, NJ: Lawrence Erlbaum Associates.
Streitz, N. A. (2007). From Human–Computer Interaction to Human–Environment Interaction: Ambient Intelligence and the Disappearing Computer. In C. Stephanidis and M. Pieper (Eds.) Universal Access in Ambient Intelligence Environments. Proceedings of the 9th ERCIM Workshop on User Interfaces for All, Königswinter, Germany, September 27-28, 2006, 3-13.