The App Checker has been developed for assessment of quality in mHealth apps for mental health. It contains three steps, altogether giving an assessment taking app information, data security and privacy and quality into account:
Before you start assessing an app make sure that you have the right information.
There are typically three sources of information about an app:
Information in the app. Sometimes the information in the app stores or on the developers website can be inadequate. In that case you may find information in the app, typically in settings, use of the app, conditions, terms and conditions of use or similar.
In this part you can tick off important information covering technical specifications, who the developer is, how the app has been developed, target group, language, operating system, price and more. You need this background information in order to decide whether or not the app is relevant to assess and use.
It is essential that you as a health professional asses whether or not the app collects personal data and processes data in a secure manner, before you recommend using it.
The assessment of security and privacy consists of 3 steps starting out with a risk assessment. The risk assessment has 8 levels, R1-R8, and the risk level of the app is decisive for the number of parameters you have to assess. If the risk level is above R1, you will be guided through a decision tree, covering whether or not personal sensitive data is collected, how data is processed and how you as a user can manage your data. If the risk level is above R5, you will also be guided through a decision tree deciding if the app should be CE-marked.
The 3 steps for assessing security and privacy:
In the risk assessment you need to look at whether or not the app collects data, which data the app collects e.g. personal data or health data and how the app is supposed to process information. Find the risk level in the assessment chart that fits the app. An app can easily fit into several levels, in that case choose the level with the highest number. The risk level determines if you need to run a check of the apps security, privacy and if it should be labelled as medical equipment (CE-marking).
If the app is placed above R1 in the risk assessment you need to assess security and privacy. In this part you will fill out a decision tree regarding security and privacy, that will help you determine if the app complies with demands related to security and privacy.
The App Checker acknowledges that apps and data collection can vary in complexity, which is why some apps will pass the test after 1 or 2 questions, while others will need to go through 8 questions.
3. Assessment of the need for CE-marking
Few apps will need to obtain compliance with CE-marking, but if the risk assessment in step 1, places the app in category R5 or upwards it is necessary to run the app through a check. The assessment is build upon The Danish Medicines Agency’s model for CE-marking and will determine if the app should be CE-marked.
In each question you can rate the app with a score from 1 to 3. In the end The App Checker will calculate an average score. To make this assessment of the quality you need to click through the functions in the app and make yourself familiar with the navigation. If you can not find the answer to a question you should rate it with 1 point. If a question is irrelevant for the app, rate it with 2 points. It will always be an individual judgement if you think the scores high enough in the categories that are most important to you. Very few app will accomplish an average score of 3 points, and on Mindapps.dk we expect an app to score higher than 1.50 if we are to recommend it.
The App Checker is a tool developed with inspiration from similar tools for assessment of quality in apps. It is a dynamic tool that will be tweaked from time to time in order to ensure that it complies with the latest developments in policy frameworks, technology and usability. Your feedback on The App Checker is most welcome.
We appreciate your input.