Systematic Monitoring of Digital Products

[Translate to English:] von einem Eyetracker generierte Heatmap
Heatmap, generated by an eye tracker and illustrating a user’s visual fixations.

Systematic monitoring is part of an internal project at the Georg Eckert Institute that aims to continuously improve the online services we provide. Methods include systematically reviewing all digital services and their use as well as developing measures to improve their usability. To this end the project team creates user profiles, which are derived from user histories and other information provided by users. Questionnaires can be used to ascertain users’ requirements. The analysis of user data is of course conducted in compliance with data protection measures in order to ensure privacy.

Once the requirements of the primary target group are known, the online services provided by the Institute can be improved accordingly. The team members are expanding their expertise in the area of usability testing during the course of the project and conducting studies on the user-friendliness of the Institute’s online services. The findings from such studies will enable the team to gauge the success of any adjustments made to online services.

The project also opens many research opportunities for the Institute, for example in the fields of artificial intelligence (AI), human-machine interaction, usability and digital humanities.

  • Aims

    The project makes a significant contribution to the continuing improvement and development of the GEI’s digital services. Its aims are as follows:

    • Analyse user groups and target user groups, review and stock-taking.
    • Conduct a needs analysis.
    • Arrange the implementation of a single sign-on authentication method.
    • Develop a user profile for research content.
    • Conduct usability tests.
    • Compose recommendations for action to improve usability.

  • Methodology

    SYSTEMATIC REVIEW OF GEI PRODUCTS AND SERVICES
    The online products and services provided by the GEI are each designed for specific user groups. In the context of systematic user monitoring it is therefore necessary to compare target and end user groups for each product in order to make decisions regarding any adjustments that need to be made and so best appeal to the relevant target group. When analysing the end user groups, utilising statistics from an internal GEI web analysis platform among other things, the focus is not on the individual steps taken by each user but rather on general behavioural patterns such as which products are accessed and how often.

    USER MONITORING
    In order to analyse groups of users and to better support them in the use of the tools, selected products or services will be accessible through a single sign-on authentication. This allows the creation of universal user profiles, which simplifies orientation for users and optimises their searches for relevant content among products and services. Users can also be provided with recommendations regarding relevant research projects, experts or appropriate publications. For such recommendations to be possible, use and user data must be captured, as well as research interests.

    USABILITY
    Usability studies aim to simplify the operation of GEI products, for example, and to increase the intuitiveness of their user interfaces. The project team have been working with the Otto von Guericke University Magdeburg to construct a mobile usability lab in which a range of usability experiments can be carried out. The GEI UsaLab will also be set up, which will enable the Institute to conduct larger scale studies, which will employ interviews with users and the use of eye tracking technology. These methods will enable the team to identify the components to which users’ attention is drawn and to measure the time taken to locate specific elements on the website.


  • Results

    The following publications have so far resulted from the project:

    • Purificato, E. 2022. ‘Beyond-Accuracy Perspectives on Graph Neural Network-Based Models for Behavioural User Profiling’. In Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization, 311–315.
    • Purificato, E., Musto, C., Lops, P., & De Luca, E. W. 2022, ‘First Workshop on Adaptive and Personalized Explainable User Interfaces’ (APEx-UI 2022). In 27th International Conference on Intelligent User Interfaces, 1–3.
    • Purificato, E., Wehnert, S., & De Luca, E. W. 2021. ‘Dynamic Privacy-Preserving Recommendations on Academic Graph Data’. In Computers, 10(9), 107.
    • Purificato, E., Manikandan, B. A., Karanam, P. V., Pattadkal, M. V., & De Luca, E. W. 2021. Evaluating Explainable Interfaces for a Knowledge Graph-Based Recommender System. Paper presented at IntRS 2021: Joint Workshop on Interfaces and Human Decision Making for Recommender Systems, Co-located with RecSys 2021 (September 25 – October 1, 2021 Amsterdam, the Netherlands).

    (Further publications pending)


Project Team

sroll-to-top