Help documentation

This page contains help documentation, including examples, for filling in the fields in a data-envelope as implemented in the Huygens Data-Envelope editor. If you want general help in creating data-envelopes, then read the guidelines

Basic workflow

Be sure to regularly save your work by pressing the Save button

Administrative information

At the top of the data-envelope there is some basic administrative information

Level 1 Basic Information

This level describes the data-envelope itself and the dataset contact person.

1.1. Title of Data-Envelope

A title for the data-envelope, NOT the dataset. Follow the format: Dataset Name Version-X Data-Envelope Version-Y.

Baptism Registers Amsterdam Version-1 Data-Envelope Version-1

The first version number refers to the version of the dataset being documented, the second version number refers to the version of the documentation itself. Typically the version number is an integer, and ‘1’ is used for the first version of the dataset or the envelope. However it can be more appropriate to use other numbering systems. For example, if a dataset already existed for a long time prior to the data-envelope being created but didn’t have a version number, then it may be more informative to use the current year. You should agree a numbering system with your team.

You can add the title in different languages if you think this is helpful to dataset users.

1.2 Contact Details

This should contain information about the contact person for the dataset.

Sometimes there is no specific contact person. In that case, instead of a name you can add a description, e.g. ‘Helpdesk’ or ‘Data management department’

1.3 Data Envelope Dates

This contains the dates for the data-envelope

1.4 Author of Data-Envelope

This should contain information about the primary author of the data-envelope itself.

1.5 Feedback and elaboration

Please use this space to provide any feedback on this section and to share any other insights or information you think would be useful. For example, you could indicate to what extent the necessary information for the data-envelope was easily available: was this a current project with experts available to answer all the questions, or is it data from fifty years ago for which the description has been pieced together from old documentation sources? You can also indicate what information sources you used to fill in the data-envelope, if you wish.

Level 2 Basic Metadata

Level 2 describes the dataset at a conceptual level. It is the ‘trailer’ that allows users to assess if the dataset is relevant to them, and as such it should be concise (preferably max 200 words). This level describes the basic dataset metadata. This includes: what the dataset contains, what the temporal and geographical coverage are, how it can be accessed etc. This section conforms to the Data Catalog Vocabulary (DCAT) standard, ensuring compatibility with machine-readable formats.

2.1 Snapshot

This contains information about what is in the dataset.

There is some discussion as to what should be included in the temporal and geographical coverage. For example, for a collection of letters the geographical coverage could refer to the locations the letters were sent to/from, or the locations discussed in the letters. Try to think about what information is most useful for users of the dataset.

Temporal coverage

This refers to the temporal coverage of the content of the dataset, NOT the dates on which the dataset was created (these should be entered in section 2.2). This could be the date range of the creation of the original items (e.g. for a collection of sixteenth century letters it could be the earliest and latest dates on which letters were sent) or it could be the period discussed in the dataset (e.g. interviews recorded discussing the Second World War in the Netherlands might cover the temporal range 1940-1944). Choose what is most relevant for users of the dataset to know.

If the temporal coverage is too complicated to explain in terms of years, then you can describe it in the Additional Notes.

2.2 Dates

These dates refer to the time span during which the dataset was created. For example, the start and end date of the project to collect and digitise the data. The date on which the dataset is published is also specified. Note: these dates could conincide with the temporal coverage, for example if the dataset contains tweets collected daily. But usually it is later; for example a dataset may contain interviews recorded this year that discuss the Second World War.

A dataset may have been worked on in several phases. If they are all part of the same project, then choose the earliest and latest dates to enter here, and if you wish you can explain in more detail in Section 2.7. If they are distinct projects - for example if your dataset is a digitised book of nineteenth century diaries that was published in the 1980s and digitised in 2023 - then choose the project that is most relevant to the dataset you are describing (in this case, the digitisation).

2.3 Creators and Contributors

In this section you can list the people and organisations who contributed to the creation of the dataset, for example by curating data, managing the project, providing funding etc. Try to use the same way of writing names across your team or institution. It is harder to find all datasets from a particular person or organisation if their name is written differently each time.

In the case of older datasets, the organisations involved may have changed their names. As a general rule, stick to the name the organisation used at the time of involvement. If you think it is helpful to the data-envelope reader, then you can add the current name in brackets. Where the organisation continues in the same role under its new name, you can use the new name and indicate that it was already involved under the old name(s). For example, the Institute for Dutch History (ING) merged with the Huygens Institute to become the Huygens Institute for History and Culture of the Netherlands. For a dataset originally published by the ING and now published by Huygens you could fill in ‘Huygens Institute for History and Culture of the Netherlands and predecessors’ for the publishing organisation.

The names of organisations may be in a different language to the language you are using when filling in the data-envelope. In this case, consider including a translation of the organisation name to help the data-envelope reader understand what sort of organisation it is.

For help with roles of creators and contributors, you can refer to the list at https://credit.niso.org/

Publishing Organisation

This is the organisation responsible for publishing the data.

Creators

Creators are responsible for the creation of the dataset. There is no strict definition of the difference between a creator and a contributor, but generally a creator makes decisions about the dataset content, while a contributor assists in creating the dataset. You can include as many creators and contributors as you like.

Contributors

Contributors aid in the creation of the dataset, for example by entering data, writing software to process the data, or providing advice from the community. There is no strict definition of the difference between a creator and a contributor. You can include as many creators and contributors as you like.

Funding Source

Here you can fill in persons, organisations or other funding sources that funded the creation, collection, or curation of the dataset.

2.4 Distribution

This section specifies where you can get the dataset or further information about the dataset, information on available downloads and on how to cite the dataset

Examples of citation formats:

For a whole dataset:

Dijkstra, G., Groskamp, N., Hoekstra, R., Koolen, M., Renkema, E., Sluijter, R., Smit, F., & Oddens, J. (2024). Entities recognised in the resolutions of the States General of the Dutch Republic (1576-1796) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.14577244

For items within a dataset (in this example letters):

A [name of correspondent] to B [name of correspondent], date [date of letter or  document], The Correspondence of William of Orange nr. [number of letter or document], date consulted. URL[number]. Example: [Willem van] Oranje to Filips van Hessen, 12-12-1560, The Correspondence of William of Orange nr. 5097, 8-2-2005. URL

2.5 Licenses and Accesses

This section tells you how you can access the dataset and under which licence(s) it may be used. Access may be direct or via a portal application. A dataset can have multiple access/licence combinations. For example, it may be browsable via an Open Access portal, while content may be downloadable via a Restricted Access portal with a licence permitting reuse.

A licence should not only be specified when access is restricted, but also when access is open. The licence determines what the user is allowed to do with the dataset. Not filling in a licence is ambiguous. In the absence of a licence, users could assume they can do anything they want, or they could assume they cannot do certain things that may actually be permitted.

The Huygens default licence is given on this page

Licensing Information

Choose a licence that suits how you want users to be able to use the material. If no licence can be given (for example if users need to make special arrangements to use the dataset), then describe how the user can find out what they need to do in the Access section below.

Note: Huygens Institute has specified a default licence that should be assumed in the absence of other information about the licence, see this information page for more details.

Access

Access Open

If the level is ‘open access’, then you can enter more information about how the open access works

Access Restricted

If the level is not open access, then you can enter more information about how access works and why it is restricted

2.6 Dataset Version and Maintenance

Version

Maintenance

This section contains information about how the content of the dataset will be maintained. Is it complete, will it be updated, will reported errors be corrected etc. It is not per se necessary to include information about technical updates to storage, servers, software etc. unless these will have a noticeable impact on users. For example, if the dataset will be unavailable for one day a month while updates are carried out, then this is a significant disruption to the user and is useful to mention. Or if technical updates will cease when a project finishes then this may increase the risk that the dataset becomes inaccessible.

Examples of maintenance plans:

Release of new data

  Reported errors in the index will be corrected. Access to cards will be unblocked either 1) after the death of the person in question is reliably reported, 2) 105 years have passed since their date of birth or 3) during periodic verifications of the data against the data held bij the CBG (Centraal Bureau voor Genealogie, Central Office of Genealogy). Otherwise the dataset is static and is not updated.

Only improvements or corrections

  This dataset will not be further enlarged. Possibly the transcriptions will be updated with improved versions.

No maintenance

  This dataset will not be further updated or enlarged.

## 2.7 Feedback and Elaboration Please use this space to provide any feedback on this section and to share any other insights or information you think would be useful

# Level 3 Data This level goes much deeper into the dataset. It should help the user use it in a correct and responsible manner. It describes how the dataset was created, and how it is structured. There is also the opportunity to describe important issues such as errors, biases, sensitivity and confidentiality.

Level 3 represents a Resource of the dataset. A Resource is something that a user can utilise to work with the dataset. A Resource can be a downloadable file, or a search portal in which the dataset can be browsed, or an endpoint for querying the data, etc. The entire level can be repeated. Each repetition represents a Resource within the dataset. Different Resources may be used, for example, to describe different files within the data. They can also be used to describe different variants that are available, for example an open version and a restricted version, if these differ significantly in how they are created or what they contain. However, you can combine multiple files or multiple versions within the same Resource. When deciding how many Resources to use to represent the dataset, consider how much of the information in Level 3 the Resources have in common, how easy it is to describe each Resource clearly and how easy it is for the user to understand what is available to them. If you are in doubt about how many Resources to use, read the guidelines.

Level 3 gives the opportunity to fill in a lot of detail, which may not be necessary for all datasets. Fill in as much as you think is useful to your users.

## 3.1 Data Resource Description This describes some basic information about the Resource, such as name, description, format and size. Depending on the dataset, some fields may be a repetition of information from Level 2, or it may be different. For example, if a dataset is split into different language files, then the Languages in Level 2 may be ‘German, Dutch and French’, while the language of the Resource representing the German section will just be ‘German’. However, if all of the Resources were a mix of the three languages then both Level 2 and all Resources would have ‘German, Dutch and French’ for their Languages.

3.2 Data Fields

These describe the individual data fields in the Resource. For example, for a spreadsheet the column and/or row headers could be described. It is also possible to link to a schema that describes the Resource structure, instead of describing the individual data fields.

If the data fields are described in the Resource file itself or they are self-explanatory, then you can skip this section.

General or overarching information about the data structure can be included in the description in Section 3.1.

External Schema

Click on the tickbox here if your dataset uses an external schema. The following fields will appear:

Data Field

This section is used to describe each data field in the dataset. It should be repeated for each data field described, by clicking on the plus icon.

Some datasets have large numbers of data fields. It is not always necessary to describe every data field. Give priority to the data fields that most users will find useful.

If you link to an external schema, then it is not necessary to fill in the individual data fields. You may still do so however if you want to call special attention to particular data fields.

Each data field can have the following information:

3.3 Data Point Examples

Typical Example

Here you can include an example of how the data typically appears in the Resource. You can copy and paste text or numerical data, or a row from tabular data, or you can describe the example (e.g. a recording of the 8pm broadcast of the daily news, recorded on 15th November 2004). If data points can be linked to, include the link so that users can examine it themselves.

Examples

A descriptive description:

a recording of the 8pm broadcast of the daily news, recorded on 15th November 2004

A copy of a data point:

<indexRecord id="saaId24681019">
<toegangsnummer>5001</toegangsnummer>
<inventarisnummer>7</inventarisnummer>
<datering>1643</datering>
<kind><voornaam>Marija</voornaam></kind>
<doopdatum>1635-01-25</doopdatum>
<kerk>Oude Kerk</kerk>
<godsdienst>Hervormd</godsdienst>
<vader><voornaam>Joost</voornaam><tussenvoegsel>van den</tussenvoegsel><achternaam>Boogaert</achternaam></vader>
<moeder><voornaam>Angnieta</voornaam><achternaam>Selijs</achternaam></moeder>
<bronverwijzing>DTB 7, p.27</bronverwijzing>
<urlScan>https://archief.amsterdam/inventarissen/inventaris/5001.nl.html#A28220000016.jpg</urlScan>
</indexRecord>

Atypical Example

Here you can include an example of known oddities or outliers in the Resource.

3.4 Errors, Noise and Redundancies

Here any errors, noise, gaps or redundancies in the Resource can be described. For example, if all data from a particular country is missing for the second half of the temporal scope, or if some dates are uncertain. Do not include biases in the data here, these should be mentioned in Section 3.7.

Example for missing data:

The transcripts cover the vast majority of these scans, the exceptions being:

the initial pages of filza (folder) 02 are not transcribed, the transcripts start at page 71;
the entire filza 9b (marked as 9-bis) is not transcribed.

The pages with missing scans are:

filza 9, pages 120r and 120v;
filza 12, pages 278r and 278v.
There are no transcripts for these pages.

3.5 External Resources

The dataset may link to external resources. For example, to pages on Wikidata describing persons, or to images stored in an image bank, or to documents in a digital archive.

E.g.: Concepts in the GTAA thesaurus https://data.beeldengeluid.nl/datasets/gtaa

3.6 Annotation and Labelling

The original data in the dataset may have been enriched by annotating or labelling it with extra information. This may be done by humans or machines. For example, experts may have labelled letters with the persons mentioned in the letters, or an algorithm may have detected persons in the letters. These annotations can be described in this section.

Annotation Characteristics

This section can be repeated for each type of annotation performed.

3.7 Social Impact, Sensitivity and Biases

Safety Consideration

Here you can answer, with yes or no, the question ‘Does the resource contain data that, if viewed directly, might be offensive, insulting, threatening, or might otherwise cause anxiety?’. If the answer is yes, then you can specify how and why. E.g. ‘The diary entries reflect the views of the time’ or ‘Descriptions may contain offensive terms’

Confidentiality

Here you can answer, with yes or no, the question ‘Is the dataset confidential?’. If the answer is yes, then you can specify what measures were taken to avoid disclosure of personal information.

Biases

Known Biases in the Resource - Identify any known biases present in this dataset, such as stereotyping, underrepresentation, overrepresentation, or misrepresentation of certain social groups. Briefly describe the nature and, if possible, the extent of these biases to help users make informed decisions and mitigate potential harms.

Steps taken to Mitigate/Reduce Biases - Describe the steps taken to reduce or mitigate biases in the dataset.

E.g. ‘This is the correspondence of a member of the royal family, so it does not offer a representative view of society at the time.’

Sensitive Attributes

For older datasets, it may be the case that they contain sensitive human attributes, but that the people involved are long dead. In this case, choose ‘Others’ and then add an explanatory text under ‘Specify’, e.g. ‘The data contains sensitive human attributes but all persons are no longer living’. This is preferable to filling in ‘None’ for the sensitive human attributes, as in that case a dataset user may then see sensitive attributes in the data and become concerned that the data-envelope is incorrect. As Lehmann et al states, “Even if such depictions are of a historical nature, it is advisable to report in the documentation that such sensitive information is contained in the dataset.”

Ethical Review

3.8 Data Provenance

The dataset may have used one or more sources in its creation. For example, a set of digital scans of letters is produced by scanning the original paper source of the letters.

Note that this is different to 3.5 External Resources. Sources in Section 3.8 were used as inputs to create the dataset. Sources in Section 3.5 are referred to within the dataset to enrich it. For example, if an archive of paper letters is digitised and the scans are made available as a digital dataset, then the original paper archive would be referenced in 3.8. If the digital scans are linked to Wikidata biographies of the persons discussed, then Wikidata would be included in 3.5 as an External Resource.

Note that this section does not refer to Data Provenance in terms of the chain of previous owners of a dataset.

For each source used, the following information may be filled in:

3.9 Digitisation

Describe the pipeline used for digitisation. If applicable, describe in what way digitisation presents another layer of selection of the whole of a collection available in a cultural heritage institution; otherwise state that all available items were digitised. If applicable, provide selection criteria and metrics that demonstrate how this additional layer of selection has influenced the transformation of the original collection or dataset into the current dataset.

Example: All residential registration cards were digitally scanned. Most were scanned with automatic feed. A number of thicker cards were manually scanned. Image optimisation was applied to the scanned images, using a light and dark profile, determined by the pixel values of the scan. As the digital scans were intended to replace the paper originals, many checks were performed on the scans. For details, see the ‘Overbrenging Amsterdamse woningkaarten 1954-1989’ report linked in Section 2.4

3.10 Feedback and Elaboration

Please use this space to provide any feedback on this section and to share any other insights or information you think would be useful

4 Uses

This level describes the uses for which the dataset was designed, what it has been used for and what are suitable uses of the dataset according to the dataset provider. It also lists unsuitable uses.

The motivating factors are very important as these will shape many decisions that have a big effect on the dataset. For example, an archive could digitally scan documents with the intention of creating digital copies and discarding the paper originals, with the intention of making the documents searchable, or with the intention of testing a new scanning tool. In the first case then the scans should be a digital copy of high quality. In the second case a lower quality is acceptable as the user only needs to be able to discover keywords in the scan. In the third case the documents themselves are not the focus of the dataset so the dataset of digital scans may not be at all representative of the dataset of paper documents, for example it may be that only robust documents that fit the scanner format were scanned.

4.1 Uses

There may be multiple types of use possible for the dataset, e.g. both research and production, so this section can be repeated per use.

Example of a suitable case:

Researching or writing about game history or the game industry

Example of an unsuitable case:

Searching for word frequencies or word use, due to OCR errors.

4.2 Use with Other Data

Datasets that are safe to use on their own may not be suitable to use with other data. For example, combining sets of data about anonymised persons may allow the persons to be identified.

It is often difficult to judge the safety of use with other data. If in doubt, select the ‘Unknown’ option. You can then leave the rest of the section empty.

4.3 Use in ML or AI systems

Datasets may or may not be suitable for use in ML or AI systems. This may be due to legal or ethical reasons, to quality or bias issues, policies of the dataset creator etc. In this section you can indicate if usage for ML/AI is allowed and offer guidelines for good usage.

It is often difficult to judge the suitability of data for ML/AI. If in doubt, select the ‘Unknown’ option. You can then leave the rest of the section empty.

4.4 Sampling

Datasets may or may not be suitable to be sampled.

It is often difficult to judge the suitability of data for sampling. If in doubt, select the ‘Unknown’ option. You can then leave the rest of the section empty.

4.5 Feedback and Elaboration

Please use this space to provide any feedback on this section and to share any other insights or information you think would be useful

Level 5 Human Perspective

All datasets are influenced by the choices of the human beings who create them, whether this is by the selection of the data, the construction of a data model, the choice of a software algorithm etc. These choices are often made against a background of a particular research question, a particular use, the experience and interests of the creators, and their social, ethnic, cultural and gender backgrounds.

In dataset documentation, embracing this human perspective is vital for various reasons. Firstly, it illuminates the biases and assumptions that may influence data collection and analysis. Secondly, it provides transparency, allowing users to understand the context in which the dataset was created and to consider how this context may affect their use of the data. Thirdly, it promotes inclusivity by recognising the diverse standpoints of dataset creators and subjects, encouraging a multiplicity of perspectives in data interpretation.

The human perspective section is related to the bias section, but it has an important difference. Biases in Section 3 are already detected and known in the dataset. The choices made due to a human perspective may result in a dataset that is well suited to the creator’s context but that has a serious bias when approached from a different domain and perspective. For example, think of an AI trained on product reviews in America by marketeers that is then applied to sentiment detection in television programmes in the Netherlands by academic researchers. Knowledge of the human perspective can help in identifying the match with different domains and uses, and in asking critical questions about the suitability of a dataset for a particular purpose.

The backgrounds of contributors can also affect the dataset. For example, if non-Dutch contributors transcribe Dutch place names, then there may be more errors than if native Dutch contributors performed the transcription. At the same time, a non-Dutch historical expert may be more successful in the transcription of historical documents than a Dutch layperson.

Avoid speculations when writing this section and stick to facts. For example, state that Dutch place names were transcribed by non-Dutch contributors. Do not add ‘so there were probably more errors’ unless this has been measured to be the case.

5.1 Human Annotators

Human annotators - people who enriched the original data with extra information such as labels - have a large influence on the results of the annotation process. For this reason, it is important to know information about the annotators that may have influenced the results. For example, native Dutch speakers may have different results when annotating documents with Dutch placenames than native English speakers. Information about the annotation instructions is also important here, as the instructions can help in standardising responses over annotators, reducing their own influence on the results.

This section focuses on who created the annotations, not on what is in the annotations. A description of what was annotated can be entered in Section 3.6.

For creators and contributors, you can fill in relevant information in Section 5.2.