Guidance to use The App Checker

Guidance 

The App Checker has been developed for assessment of quality in mHealth apps for mental health. It contains three steps, altogether giving an assessment taking app information, data security and privacy and quality into account:

 

Where can you find information about an app?

Before you start assessing an app make sure that you have the right information.

There are typically three sources of information about an app:
Information in app stores like App Store or Google Play. Any serious app developer should supply thorough information about an app in the app store. This information can consist of a description about content,version, functionality and as a minimum a privacy policy or terms and conditions of use. They will describe how data is collected and processed, if there is any agreement with third parties and how you can receive your data or get data deleted after end of use.Typically you can find links to these policies under Information in the app store.

Information on the developers website. Some developers prefer to supply extensive information on their website. In that case there should be a link to the developers website in the app store. The information you need to seek out will be the same: Information about the app, a description about content, functionality and a privacy policy.

Information in the app. Sometimes the information in the app stores or on the developers website can be inadequate. In that case you may find information in the app, typically in settings, use of the app, conditions, terms and conditions of use or similar.

 
NB: If the developer does not have a privacy policy you should consider not to use the app.

 

Step 1 in The App Checker: Information about the app.

In this part you can tick off important information covering technical specifications, who the developer is, how the app has been developed, target group, language, operating system, price and more. You need this background information in order to decide whether or not the app is relevant to assess and use.

Step 2 in the App Checker: Assessment of security and privacy

It is essential that you as a health professional asses whether or not the app collects personal data and processes data in a secure manner, before you recommend using it.

The assessment of security and privacy consists of 3 steps starting out with a risk assessment. The risk assessment has 8 levels, R1-R8, and the risk level of the app is decisive for the number of parameters you have to assess. If the risk level is above R1, you will be guided through a decision tree, covering whether or not personal sensitive data is collected, how data is processed and how you as a user can manage your data. If the risk level is above R5, you will also be guided through a decision tree deciding if the app should be CE-marked.

 

The 3 steps for assessing security and privacy:

  1. Risk assessment

In the risk assessment you need to look at whether or not the app collects data, which data the app collects e.g. personal data or health data and how the app is supposed to process information. Find the risk level in the assessment chart that fits the app. An app can easily fit into several levels, in that case choose the level with the highest number. The risk level determines if you need to run a check of the apps security, privacy and if it should be labelled as medical equipment (CE-marking).

  1. Assessment of data security and privacy

If the app is placed above R1 in the risk assessment you need to assess security and privacy. In this part you will fill out a decision tree regarding security and privacy, that will help you determine if the app complies with demands related to security and privacy.

The App Checker acknowledges that apps and data collection can vary in complexity, which is why some apps will pass the test after 1 or 2 questions, while others will need to go through 8 questions.

3. Assessment of the need for CE-marking
Few apps will need to obtain compliance with CE-marking, but if the risk assessment in step 1, places the app in category R5 or upwards it is necessary to run the app through a check.  The assessment is build upon The Danish Medicines Agency’s model for CE-marking and will determine if the app should be CE-marked.

 

Step 3 in the App Checker: Assessment of quality

Now you need to do the final assessment of the quality of the app. This part of the App Checker takes you through 4 categories, each containing 3 questions:

 

  1. Background information
  2. Clinical quality
  3. Functionality
  4. Usability

 

In each question you can rate the app with a score from 1 to 3. In the end The App Checker will calculate an average score. To make this assessment of the quality you need to click through the functions in the app and make yourself familiar with the navigation. If you can not find the answer to a question you should rate it with 1 point. If a question is irrelevant for the app, rate it with 2 points. It will always be an individual judgement if you think the scores high enough in the categories that are most important to you. Very few app will accomplish an average score of 3 points, and on Mindapps.dk we expect an app to score higher than 1.50 if we are to recommend it.

Give us your feedback

The App Checker is a tool developed with inspiration from similar tools for assessment of quality in apps. It is a dynamic tool that will be tweaked from time to time in order to ensure that it complies with the latest developments in policy frameworks, technology and usability. Your feedback on The App Checker is most welcome.

 
Email us and tell us how it is to use the App Checker on mindapps@rsyd.dk.

Was the article helpful to you?

Up
Down

Thanks for your feedback

Tell why you liked or disliked this page.

Thank you

We appreciate your input.

Subscribe to our newsletter

Psykiatrien i Region Syddanmark Region Syddanmark CoLab Denmark