Package: HighFrequencyChecks 0.5.0

Edouard Legoupil

HighFrequencyChecks: High Frequency Checks

During the data collection, a series of automatic check, aka: High Frequency checks, are required. The functions shared here are useful during the data collection process to check periodicallyxfor possible errors, and will provide meaningful inputs to the enumerators. All these functions do not have to be ran at the same period of time. They are provided there to help data supervisor to build reports. This work is an adaptation of a Stata Package from [Innovations for Poverty Action](https://github.com/PovertyAction/high-frequency-checks).

Authors:Edouard Legoupil [aut, cre], Yannick Pascaud [aut], UNHCR [cph]

HighFrequencyChecks_0.5.0.tar.gz
HighFrequencyChecks_0.5.0.zip(r-4.7)HighFrequencyChecks_0.5.0.zip(r-4.6)HighFrequencyChecks_0.5.0.zip(r-4.5)
HighFrequencyChecks_0.5.0.tgz(r-4.6-any)HighFrequencyChecks_0.5.0.tgz(r-4.5-any)
HighFrequencyChecks_0.5.0.tar.gz(r-4.7-any)HighFrequencyChecks_0.5.0.tar.gz(r-4.6-any)
HighFrequencyChecks_0.5.0.tgz(r-4.6-emscripten)
manual.pdf |manual.html
DESCRIPTION |NEWS
card.svg |card.png
HighFrequencyChecks/json (API)

# Install 'HighFrequencyChecks' in R:
install.packages('HighFrequencyChecks', repos = c('https://unhcrverse.r-universe.dev', 'https://cloud.r-project.org'))

Bug tracker:https://github.com/edouard-legoupil/highfrequencychecks/issues

Pkgdown/docs site:https://edouard-legoupil.github.io

On CRAN:

Conda:

data-quality-checksresearch-data-managementresearch-toolsurvey

3.61 score 1 stars 27 scripts 32 exports 184 dependencies

Last updated from:92c85ccda6. Checks:5 WARNING, 2 OK, 2 ERROR. Indexed: no.

TargetResultTimeFilesSyslog
linux-devel-x86_64WARNING244
source / vignettesOK324
linux-release-x86_64WARNING237
macos-release-arm64ERROR226
macos-oldrel-arm64ERROR109
windows-develWARNING196
windows-releaseWARNING181
windows-oldrelWARNING179
wasm-releaseOK215

Exports:%>%assessmentDailyValidSurveysassessmentDurationassessmentDurationOutliersassessmentInterviewTimeassessmentProductivityassessmentTrackingSheetenumeratorErrorsSummaryenumeratorIsLazyenumeratorProductivityenumeratorProductivityOutliersenumeratorSurveysConsentenumeratorSurveysDurationisInterviewAtTheSamplePointisInterviewCompletedisInterviewInTheCorrectSiteisInterviewTooShortisInterviewTooShortForTheHouseholdSizeisInterviewWithConsentisSurveyEndBeforeItStartsisSurveyMadeInTheFutureisSurveyOnMoreThanADayisSurveyStartedBeforeTheAssessmentisuniquerespondantIDDuplicatedisuniquerespondantIDMissingrun_appsurveyBigValuessurveyDistinctValuessurveyMissingValuessurveyOtherValuessurveyOutlierssurveySmallValues

Dependencies:AsioHeadersaskpassattemptbackportsbase64encBHbitbit64blobbookdownbootbroombslibcachemcallrcellrangerclassclicliprcodetoolscolorspacecommonmarkconfigconflictedcowplotcpp11crayoncurldata.tabledatawizardDBIdbplyrDescToolsdigestdplyrdtplyre1071evaluateExactexpmfarverfastmapflextablefontawesomefontBitstreamVerafontLiberationfontquiverforcatsfsgarglegdtoolsgenericsggplot2ggrepelggtextggwordcloudgldgluegolemgoogledrivegooglesheets4GPArotationgridExtragridtextgtablehavenherehighrhmshtmltoolshttpuvhttridsinsightisobandjpegjquerylibjsonliteknitrkobocruncherlabelinglaterlatticelifecyclelikertlitedownlmomlubridatemagickmagrittrmarkdownMASSMatrixmemoisemimemnormtmodelrmvtnormnlmeNLPofficedownofficeropensslopenxlsxoteloutlierspagedownpillarpkgconfigplyrpngprettyunitsprocessxprogresspromisesproxypspsychpurrrR6raggrappdirsRColorBrewerRcppreadrreadxlrematchrematch2reprexreshape2riddlerlangrmarkdownrootSolverprojrootrstudioapirvestrvgS7sassscalesselectrservrshinyshinydashboardshinyWidgetsshowtextshowtextdbsjlabelledslamSnowballCsourcetoolsspstringistringrsyssysfontssystemfontstextshapingtibbletidyrtidyselecttidyversetimechangetinytextmtzdbunhcrdownunhcrshinyunhcrthemesutf8uuidvctrsviridisLitevroomwebsocketwithrxaringanxfunXlsFormUtilxml2xtableyamlzip

Package Functions
Demo data | Pro-active actions - Enumerators rigorous work standards | Interview Duration | Interview Duration Outliers | Picked up less than a certain number of answers for select multiple questions | Are Interviews Completed? | Do Interviews include Consent? | Is there Missing Values in the reponse? | Number of distinct values (not missing) per fields | Other distinct values (not missing) per fields other with count | Response Outliers | Abnormally High Values | Abnormally Low Values | Enumerator Errors Summary | Pro-active actions - Enumerators productivity | Summary of daily average productivity | Daily number of filled forms per attempt output | Daily number of filled forms per time in the day | Percentage of non-completed interviews by enumerator | Interview Duration per Enumerator | Number of Interview by enumerator | Enumerators with very low or high productivity | Corrective actions - Correct set-up of data collection devices and encoding of the forms | How many Attempt for each SampleRespondant ID? | Are there unique respondantID Missing? | Is there any survey interview lasting on more than a day? | Is there any survey interview with end date time before start time? | Is there any survey interview with start date after today? | Corrective actions - Data collected according the plan | Is there any survey interview starting before official first day? | Are there interviews appearing too short? | Are there interviews appearing too short based on household Size? | Are interviews taking place in the expected site? | Are interviews taking place in the expected point? | Overall tracking sheet | run_app

Last update: 2023-09-21
Started: 2023-09-20

Automating Report Generation and Notification
Automating Data Quality Monitoring Report | Create a public github repo | Set up the manifest | Publish the report to Rstudio Connect from Github | Set up your kobotoolbox API key within Rstudio Connect | Set up report frequency generation and sending it to your email | Use power automate to forward automatically the email with the report to the data supervisor | Doing some of the available checks on the individual loops | Doing a check by enumerator or not | Periodic checks (daily for example) on the data collected | Changing the purpose of a function

Last update: 2023-09-20
Started: 2023-09-20

Configuring High Frequency Check Report Template
Measuring data collection quality | Specific variables to be controlled | Setting up tablet | Process configuration | Loading survey dataset (microdata) | Configuring numeric questions | Configuring categoric questions | Configuring sampling plan | Configuring geodata for the surveyed area | Report Configuration | Corrective actions | Correct set-up of data collection devices and encoding of the forms | Respondent ID | Configuration of dates on device | Data collected according the plan | Completed interviews | Interviews made before the first day of data collection | Enumerators who made a survey below r minimumSurveyDuration minutes | Recorded site name for each interview matches the name of the location | Recorded locations for each interview within a r buffer meters buffer from a sample point | Tracking sheet per site | Pro-active actions | Enumerators rigorous work standards | Durations of Interviews | Responses with outliers | Numeric value above a certain threshold | Enumerators who pick up less than a predifined number of answers per specific questions: | Check percentage of “don’t know” and refusal responses by the enumerator. | Number of other distinct values (for the questions with a possibility of other) | Enumerator productivity | How many completed interview per day? | How many attempted interview per day and obtained consent? | Percentage of survey per consent status by enumerator | Average interview duration by enumerator | Number of surveys per day by enumerator | Enumerators with productivity significantly different from the average (low or high)

Last update: 2023-09-20
Started: 2023-09-20

Readme and manuals

Help Manual

Help pageTopics
Daily number of filled forms per consent statusassessmentDailyValidSurveys
Compute the average and total time for the surveysassessmentDuration
Report the outlier durations for the surveysassessmentDurationOutliers
Daily number of filled forms per consent statusassessmentInterviewTime
Summary of daily average productivityassessmentProductivity
Overall tracking sheetassessmentTrackingSheet
Create a dashboard displaying the number of errors by enumeratorsenumeratorErrorsSummary
Check the enumerators who pick up less than X answers per specific questionenumeratorIsLazy
Check the number of Interview by enumeratorenumeratorProductivity
Check the enumerators with very low or high productivityenumeratorProductivityOutliers
Percentage of non-completed interviews by enumeratorenumeratorSurveysConsent
Check the average interview duration by enumeratorenumeratorSurveysDuration
GIS check surveys if fall without Xm radius from a sampled pointisInterviewAtTheSamplePoint
Check that all interviews were completedisInterviewCompleted
GIS check surveys for siteisInterviewInTheCorrectSite
Check that the duration of each interview is more than a thresholdisInterviewTooShort
Check that the duration relative to the household size of each interview is more than a thresholdisInterviewTooShortForTheHouseholdSize
Check that all surveys have consentisInterviewWithConsent
Surveys where end date/time is before the start date/timeisSurveyEndBeforeItStarts
Surveys that have start date/time after system dateisSurveyMadeInTheFuture
Surveys that do not end on the same day as they startedisSurveyOnMoreThanADay
Surveys that show start date earlier than first day of data collectionisSurveyStartedBeforeTheAssessment
Duplicates in unique IDisuniquerespondantIDDuplicated
Missing unique IDisuniquerespondantIDMissing
Run the Shiny Applicationrun_app
Report the values greater than a specified value per specified fieldssurveyBigValues
Number of distinct values (not missing) per fieldssurveyDistinctValues
Report the percentage of missing values (NA) per fieldssurveyMissingValues
List of other distinct values (not missing) per fields other with countsurveyOtherValues
Report the outlier values for all numerical fieldsurveyOutliers
Report the values lower than a specified value per specified fieldssurveySmallValues