Day 3: Humanitarian information systems and responsible…

1Designing an information system

An information system starts from the decisions it must inform: who decides what, when, with what information? From there come the indicators, sources (assessments, 4W, monitoring, secondary data), frequency and products (dashboards, maps, analysis notes). Shared standards (P-codes, HXL to tag tables, common operational datasets on HDX) make it possible to combine data from several organisations.

The golden rule: every piece of data collected must be used. Overlong forms tire respondents and degrade quality.

2Quality, bias and limitations

Humanitarian data are often incomplete: inaccessible areas, convenience samples, hard-to-reach groups (people with disabilities, minorities). Quality is assessed in terms of validity, reliability, precision, timeliness and integrity, and biases are documented.

Communicating limitations does not weaken the analysis: it prevents decisions based on false certainty. A half-page methodological note accompanies every important product.

3Responsible data

The IASC Operational Guidance on Data Responsibility (2023) and the ICRC Handbook on Data Protection in Humanitarian Action set the principles: defined purpose, minimisation, informed consent or another legitimate basis, security, controlled sharing and limited retention.

Impact assessment before any risky processing.
Information-sharing protocols at country and cluster level, classifying data by sensitivity.
Aggregation and thresholds for public products, so that nobody can be identified.

In-depth course

Detailed methods, a complete worked example, tools, common mistakes and interview questions.

4Architecture of an information system for a cluster or a programme

An information system is not a piece of software. It is the organised set of data flows, people, rules, tools and products that enables a response to make informed decisions. The IASC Reference Module for Cluster Coordination at Country Level makes information management a core function of every cluster; in practice, the quality of this function depends less on the tool chosen than on the clarity of rules and roles.

The components to design

Component	Questions to settle	Deliverable
Information needs	Which decisions, by whom, by when?	Validated information needs matrix
Data flows	Who produces what, how often, through which channel, for whom?	Data flow diagram and reporting calendar
Standards	Which P-codes, which indicator definitions, which activity lists, which HXL hashtags?	Data dictionary and indicator guidance
Tools	Collection, storage, consolidation, visualisation; hosting and costs	Reasoned choice of tools and continuity plan
Roles	Who checks, who validates, who publishes, who has access to what?	Responsibility and access rights matrix
Products	Which products, for which audience, how often?	Information products catalogue
Data responsibility	What sensitivity, what sharing rules, what retention periods?	Data and information sensitivity classification, information sharing protocol, impact assessments

The timeline of a cluster system within the humanitarian programme cycle

Monthly: partners' 4W or 5W, cluster dashboard, response plan indicator monitoring submitted to OCHA.
Quarterly: gap analysis, presence map, review of indicators with the cluster's Strategic Advisory Group.
Annual: contribution to the Humanitarian Needs Overview and the response plan (now often merged into a single Humanitarian Needs and Response Plan), calculation of people in need and severity, target setting, contribution to the multi-sector needs assessment.
Event-driven: alerts and rapid assessments when a new shock occurs, with lighter, faster products.

The information products catalogue

Product	Audience	Frequency	Dissemination level	Responsible
Cluster dashboard	Partners, OCHA, donors	Monthly	Public (aggregated at district level)	Cluster IMO
Operational presence map	Partners, coordination	Monthly	Public	Cluster IMO
Gap analysis by municipality	Strategic Advisory Group	Quarterly	Restricted to members	IMO and coordinator
Consolidated 4W dataset	Active partners	Monthly	Restricted	Cluster IMO
Situation note on a shock	Coordination, donors	Event-driven	Depending on content	Coordinator with the IMO

Choosing tools without locking yourself in

The criteria are: partners' capacity (connectivity, skills), sustainable cost (licences, hosting), data hosting and jurisdiction, the ability to export in open formats, and maintenance after the person who built the system has left. Common combinations pair KoboToolbox or ODK for data collection, ActivityInfo or a locked Excel template for the 4W, a shared storage space with permission management, Power BI or Excel for dashboards, QGIS for maps and HDX for publishing aggregated datasets.

Sustainability test: if the IMO who designed the system leaves the mission tomorrow, can a successor produce the next dashboard in less than a week from the existing documentation? If the answer is no, the system is not finished.

Human resources

A cluster active in a large-scale crisis needs at least one full-time IMO, often supported by an assistant; sub-national levels need focal points. The IMO role is often underestimated in budgets: the position must be defended from the design stage, with the list of expected products and the corresponding workload.

5Assessing data quality: method and assessment grid

The five classic dimensions of data quality (validity, reliability, precision, timeliness, integrity) are well known; what is often missing is a method for assessing them regularly and in a documented way. The data quality assessment (DQA), practised in particular by development cooperation agencies and health programmes, provides that method.

The dimensions and their control questions

Dimension	Question	Example of a check
Validity	Do the data measure what they claim to measure?	Does “people with access to water” measure actual access or the presence of a nearby water point?
Reliability	Does the same method produce the same results from one time to the next and from one data collector to another?	Written definitions, training, comparison between enumerators
Precision	Are the level of detail and the margin of error sufficient for the decision?	A 10% margin is acceptable for prioritisation but insufficient to track a 5-point change
Timeliness	Are the data available in time and recent enough?	Time lag between the end of the period and the availability of the figure
Integrity	Are the data protected against manipulation and deliberate errors?	Segregation of duties, traceability of changes
Completeness	Have all expected units reported?	Reporting rate of partners or facilities

Verification by recount: the verification factor

For a monitoring indicator (people trained, consultations, kits distributed), a sample of sites and months is selected, figures are recounted from primary sources (registers, signed lists, waybills) and compared with the reported figure. The verification factor is the ratio between the recounted figure and the reported figure.

Site	Reported figure	Recounted figure	Factor	Finding
Health centre A	1,240	1,228	0.99	Negligible discrepancy
Health centre B	980	742	0.76	Over-reporting: follow-up consultations counted as new
Health centre C	615	690	1.12	Under-reporting: night register not taken into account
Total	2,835	2,660	0.94	Overall over-reporting of 6%

A range of 0.90 to 1.10 is often considered acceptable; outside it, the cause is investigated (misunderstood definition, unsuitable form, pressure to meet targets) and the process is corrected, not just the figure.

Method for a data quality assessment

Select the indicators to be assessed: those reported to the donor and those underpinning important decisions. Generally three to five.
Map the flow of each data item: from the primary source to the final report, with every step of data entry, transfer and aggregation. Errors arise at transfer points.
Assess the system: written definitions, standard forms, training, supervision, checks, storage, roles.
Verify the figures by recounting a sample of sites and periods.
Draft the improvement plan: findings, actions, responsible persons, deadlines, follow-up at the next assessment.

A data quality assessment does not look for culprits: it looks for the points in the flow where information becomes distorted. Presented as a punitive check, it produces doctored data rather than accurate data.

Triangulation day to day

Without waiting for a formal assessment, the IMO systematically cross-checks sources: reported distributions against stock releases; people reached against the baseline population; survey results against monitoring data; trends in one source against trends in another (a rise in nutrition admissions that is or is not consistent with the lean season calendar). An inconsistency is not a proven error; it is a question to be asked.

6Bias, limitations and confidence levels: identifying and communicating them

Humanitarian data are collected in conditions that produce systematic bias: limited access, short timeframes, intermediaries, respondents' fear, expectations of assistance. The professional does not claim to eliminate them; they identify them, reduce them where possible and declare them so that the reader can take them into account.

Typology of the most common biases

Bias	How it shows	Possible mitigation
Access bias	Only safe areas close to roads are covered	Remote key informants, phone interviews, local partners on the ground, satellite data; explicit declaration of missing areas
Selection bias	Enumerators interview households that are present during the day, visible, close to the centre	Random selection, return visits, varied visiting hours
Gatekeeper bias	Site leaders or authorities choose the respondents	Verified lists, selection by the team, interviews out of the presence of leaders
Social desirability and expectation bias	People exaggerate or downplay their needs depending on what they think they will receive	Clear information that there is no link to assistance, factual questions rather than perceptions, triangulation
Invisibility bias	Persons with disabilities, older people, minorities, women on their own under-represented	Washington Group questions, female enumerators, individual interviews, complementary targeted sampling
Survivorship bias	Data on displaced people only cover those who arrived	Mention in the analysis; data on areas of origin where they exist
Reporting bias	Incidents are reported where services or actors are present	Never read a map of reported incidents as a map of actual incidents

Reporting bias is the most common trap in protection data: a rise in recorded cases may reflect the opening of a new service or greater trust, not an increase in violence. Conversely, the absence of reports in an area proves nothing.

A confidence scale for each finding

Level	Criteria	Recommended wording
High	Representative method, recent data, several corroborating sources	“44% of households…”
Medium	Partial method or data more than six months old, or a single reliable source	“According to the February assessment, around 4 in 10 households…”
Low	Few key informants, areas not covered, contradictory sources	“Available information suggests… to be confirmed”

Template for a half-page methodological note

Purpose and questions: what the product is for.
Sources and dates of data collection.
Method: population covered, sampling, sample size, level of representativeness, weighting.
Coverage: areas and groups covered, areas and groups not covered.
Known limitations and biases, with their likely direction (overestimation, underestimation).
Confidence level for each main finding.
Contact for questions about the data.

This note protects the analyst as much as the reader: a figure taken out of context in an advocacy document can always be traced back to its methodological note.

7Governing data responsibility across a response

The IASC Operational Guidance on Data Responsibility in Humanitarian Action, revised in 2023, defines data responsibility as the safe, ethical and effective management of personal and non-personal data. It sets out principles (including a people-centred approach, do no harm, personal data protection, defined purpose, proportionality, quality, security, transparency and accountability) and actions to be taken at three levels: the response as a whole, each cluster or sector, and each organisation.

Key actions at response level

Map the data ecosystem: which organisations collect which data, on whom, with which tools, where are they stored, who shares them with whom? This mapping reveals redundant data collection and high-risk flows.
Establish a common data and information sensitivity classification.
Adopt a response-wide information sharing protocol, under the auspices of the information management working group, with the sensitivity classification annexed.
Conduct impact assessments for high-risk processing (biometric registration, case management systems, incident mapping).
Plan for data incident management: definition, notification, response, lessons learned.

Example of a sensitivity classification

Level	Definition	Examples	Sharing rules
Low or public	Disclosure carries no reasonable risk of harm	4W aggregated at district level, severity maps, baseline population figures	Can be published (HDX, websites)
Moderate or restricted	Limited risk, low impact	4W by municipality and by site, assessment findings by settlement	Shared among humanitarian partners on request, no republication
High	Risk of significant harm to individuals, groups or organisations	Precise locations of IDP sites in conflict zones, geolocated protection incident data	Limited sharing, specific purpose, written agreement, fixed level of aggregation
Severe or confidential	Risk of serious harm, including to life	Lists of names, case management data, information on survivors of violence, biometric data	No sharing beyond the purpose and beyond consent; named, minimal access

Conducting a data protection impact assessment

Describe the processing: purpose, data, data subjects, tools, flows, recipients, retention periods.
Check necessity and proportionality: is each data item necessary? Is there a less intrusive solution?
Identify the risks to people: unauthorised access, misuse by an authority or armed group, exclusion by error, stigmatisation, loss of trust. For each, likelihood and severity.
Define mitigation measures: technical (encryption, pseudonymisation), organisational (roles, training), contractual (agreements).
Decide: acceptable residual risk, processing modified or abandoned. The decision is signed off by a manager at the appropriate level.
Review when the context or the processing changes.

An impact assessment that systematically concludes that the risk is acceptable is not an assessment: it is a formality. Good assessments regularly lead to removing a variable, reducing a level of detail or forgoing a data share.

The special case of protection information

The Protection Information Management (PIM) framework, developed by a group of humanitarian organisations, distinguishes several systems: population data, protection monitoring, case management, response monitoring, and security and situational awareness. Each has its own purposes and its own rules. The ICRC Professional Standards for Protection Work devote an entire chapter to managing sensitive data and information. Mixing protection monitoring data and case management data in the same dataset is a classic design error.

8Full example: building a protection cluster's information system in eight weeks

Fictional context: in the province of Bas-Ombré (Valdorie), an offensive has displaced 140,000 people in a few months. The recently activated protection cluster has 17 partners, including 9 national organisations. There is no common 4W and no sharing rules; three organisations each run their own protection monitoring with different questionnaires; an advocacy platform is calling for a public map of incidents. An IMO is recruited for six months.

Weeks 1 and 2: diagnosis

Interviews with the cluster coordinator, the leads of the areas of responsibility (child protection, gender-based violence, mine action), 12 partners and the OCHA IMO.
Mapping of the ecosystem: 23 data flows identified, 5 of them assessed as high risk (lists of names sent by email to local authorities for registration, incident tables with names shared on a group messaging app).
Information needs matrix: 6 priority decisions, including the deployment of mobile teams, the choice of sites for safe spaces, and advocacy on freedom of movement.

Weeks 3 and 4: standards and rules

The cluster adopts a list of 14 standard activities, a single 4W with P-codes and disaggregation by sex, age and disability, and a common community-level protection monitoring questionnaire (key informants, no individual data), which replaces the three existing questionnaires. The sensitivity classification is validated at a cluster meeting:

Dataset	Level	Form in which shared	Recipients	Retention
Protection 4W	Low (district); moderate (municipality)	Aggregated	Public at district level; partners at municipality level	Duration of the response
Protection monitoring (key informants)	Moderate	Indicators aggregated by municipality and by month	Cluster partners	Three years, then aggregated archiving
Reported protection incidents	High	Trends by district and by category, no precise location, cells below 5 merged	Restricted group (cluster coordination, OCHA, OHCHR); trends for advocacy	Two years for detailed data
Case management (child protection, GBV)	Severe	No individual data; aggregated statistics produced by each organisation	Managing organisation only	According to case management procedures
List of IDP sites with coordinates	High in frontline areas	Precise coordinates for internal planning; not published	Operational partners with an agreement	Updated monthly

Week 5: impact assessment of the public incident map

The assessment concludes that the proposed map (geolocated points, incident category, date) would make it possible to identify villages, and in some cases families, exposed to reprisals; and that 81% of recorded incidents come from three municipalities where mobile teams operate, which would make the map misleading. Decision: no point map. Instead, a quarterly table of trends by district and by category, accompanied by a qualitative note and the reporting bias disclaimer.

Weeks 6 and 7: tools and training

Common form deployed in an account held by the cluster lead agency, with per-partner permissions (each partner sees its own submissions, the IMO sees everything); online 4W with a closed list of P-codes; two-day training for 31 focal points, including half a day on data responsibility; data incident management procedure drafted and disseminated. The lists of names sent to the authorities are replaced by figures aggregated by site, following advocacy by the coordinator backed by the adopted classification.

Week 8: first products and review

First dashboard: 16 of the 17 partners reported; 38,000 people reached by at least one protection activity (de-duplication by maximum at municipality level); two severity 4 municipalities with no partner present. The cluster's Strategic Advisory Group decides to redeploy two mobile teams. The end-of-period review identifies the points to consolidate: two national partners lack capacity for online data entry (support from a part-time IMO assistant budgeted under the pooled fund), and a formal data sharing agreement is needed with the GBV area of responsibility.

Here the system produces less data than before (one questionnaire instead of three, no lists of names shared), but data that are more useful and less dangerous. This is often the sign of a well-designed system.

9Design mistakes, donor and auditor expectations, adapting to the context

Common design mistakes

Mistake	Consequence	Prevention
Starting with the tool (“we are going to build a Power BI dashboard”)	An elegant product that serves no decision	Start with the information needs matrix
System dependent on a single person	Collapse when the IMO leaves	Documentation, commented scripts, a second person trained
Proliferation of reporting requests to partners	Reporting fatigue, sloppy data, disengagement	A single form, a fixed calendar, systematic feedback of analyses to partners
No sensitivity classification	Sharing decisions made case by case, under pressure	Classification and protocol adopted early, at a coordination meeting
Confusion between protection information systems	Case management data exposed in trend analyses	Separate datasets, distinct purposes, reference to the PIM framework
Data collection without feedback to communities	Loss of trust, refusal to participate	Sharing findings back in an appropriate form, while respecting confidentiality

What donors, evaluators and auditors examine

Accountability of the function: donors funding cluster IMO positions expect regular products, a partner reporting rate and contributions to planning documents.
Demonstrated quality: data quality assessments conducted, verification factors, corrective actions followed up.
Compliance: the organisation's data protection policy, impact assessments for high-risk processing, signed data sharing agreements, a data incident register. Institutional audits by major donors and partner assessments by United Nations agencies increasingly include these elements.
Use: evaluations, including Inter-Agency Humanitarian Evaluations, examine whether information influenced resource allocation and prioritisation.
Inclusion: consideration of persons with disabilities, women and marginalised groups in the design of data collection, in line with the Core Humanitarian Standard (CHS) commitments on participation and non-discrimination.

Adapting the system to the context

Context	Design priorities
Sudden-onset crisis	A minimal system within days: simple 3W, map of affected areas, contact list, interim sharing rules adopted at the first meeting; full standards come later
Protracted crisis	Stable indicators, archiving, time series, gradual transition to national systems where possible and safe
Camp settings	Close coordination with camp management and registration; avoid each organisation re-enumerating the population
Urban setting	Integration of data from municipal authorities and public services, particular attention to the visibility of displaced people
Remote management	Tools that work offline and on low connectivity, third-party monitoring, automated checks, even more rigorous documentation
Conflict with parties hostile to humanitarians	Strict data minimisation, data hosted out of the parties' reach, emergency wipe plan, impact assessment for any new data flow
National partners with limited capacity	Simple tools, training and accompaniment, funding for IM positions within partners, recognition of their data in products; in line with localisation commitments, do not confine them to data collection

10Interview questions and key references

Interview questions

1. “You arrive as IMO of a newly activated cluster. What do you do in the first two weeks?”

A good answer describes the diagnosis: meetings with the coordination team and partners, inventory of existing data, mapping of data flows, identification of priority decisions. It proposes quick minimal products (contacts, 3W, map) while preparing the standards, and mentions sharing rules from the outset.

2. “How do you assess the quality of the figures partners send you?”

The candidate cites the dimensions of quality, automated checks of the 4W, triangulation with other sources, recounting on a sample and the verification factor, and emphasises improving the process rather than sanctioning.

3. “An advocacy organisation asks for a public map of protection incidents. What do you reply?”

A good answer refers to the impact assessment, the risks of re-identification and reprisals, and reporting bias, and proposes alternatives: trends aggregated by district and by period, a qualitative note, disclaimers. It mentions the sensitivity classification and collective decision-making within the cluster.

4. “What is data responsibility, and how does it differ from data protection?”

Data protection concerns personal data; data responsibility also covers sensitive non-personal data (demographically identifiable data, locations of groups, operational information), and includes the effectiveness and ethics of their use. A good answer cites the IASC guidance and the three levels of action.

5. “How do you present the limitations of an analysis to a coordinator who wants a simple figure?”

A good answer gives a figure with its confidence level and a one-sentence limitation, explains what the limitation concretely changes for the decision, and proposes how to reduce the uncertainty if the decision warrants it.

6. “How do you make an information system sustainable after you leave?”

Documentation (data dictionary, indicator guidance, procedures), automation through commented scripts, training a back-up, choosing maintainable tools, involving national partners and a formal handover.

Key references

Operational Guidance on Data Responsibility in Humanitarian Action: IASC (2023).
Data Responsibility Guidelines and guidance notes from the Centre for Humanitarian Data: OCHA.
Reference Module for Cluster Coordination at Country Level: IASC.
Guidance on the humanitarian programme cycle and the Joint Intersectoral Analysis Framework (JIAF): IASC and OCHA.
Protection Information Management (PIM) framework: inter-agency PIM Initiative.
Professional Standards for Protection Work: ICRC, chapter on managing data and information.
Handbook on Data Protection in Humanitarian Action: ICRC and Brussels Privacy Hub.
Core Humanitarian Standard on Quality and Accountability (CHS).
Principles for Digital Development.
Data quality assessment tools and guides used by development cooperation agencies and public health programmes.