Collecting data looks easy: add one more column, ask one more question. The problem arrives later. Fixing wrongly collected data costs far more than fixing a wrongly written rule, because a rule sits in one place while data accumulates everywhere.
This lesson takes on the ground automation is built upon: customer data. In the previous lesson you saw that a large part of faults come out of the input side. Here we set that side up correctly from the start, which means which fields get collected, how they get written and where they live.
When data is collected wrongly the problem does not show at once. It comes out months later, in bulk, and calls for cleaning up backwards. The aim of this lesson is that you build the order that makes such a cleanup unnecessary in the first place.
No prerequisites, though having the process map from lesson one at hand makes it easier to see which field is genuinely used.
An automation runs on the data reaching it. However correctly the rule is written, if the incoming data is broken the outgoing result is broken too. That is why data collection is not a part of the automation but the ground it stands on.
The cost of broken data grows with time. When one record goes in wrong in the first week, correcting it takes a moment. When the same mistake carries on for a year, correcting it calls for reviewing every record, and that work often goes undone while trust in the data falls.
When a field goes in wrong, every flow running on that field is touched. Messages, reports, invoices and reminders all feed from the same source.
When a mandatory field arrives empty, the flow either stops or skips that step quietly. The second is the more dangerous, because nobody notices.
If the same information sits in three places in three shapes, which one is right stays arguable. No report can be pulled, and one that is pulled cannot be trusted.
The whole of a data order rests on three rules. Where these three hold, the remaining detail stays manageable; where they do not, swapping tools rescues nothing.
A customer wants to be one row in the system. Once the same person is opened twice, the history splits, which record is current becomes unclear, and messages go out twice.
The same information wants to be written the same way everywhere. A single spelling is chosen for phone, date, city name and currency, and applied at every point of entry.
Each field wants one correct place. If the same information is held in two systems, which one gets updated is written down, and the other feeds from it.
The three rules hold each other up. Without one format, deduplication does not work, because the same person looks like two people under two spellings. Without one source, one row is not kept either, because every system updates its own copy.
The measure in choosing fields is use, not curiosity. Before collecting a field, ask this: in which task, at which step, will this information be used? Fields with no answer do not get collected. Fields collected in case they come in handy usually stay empty, and where they do not stay empty they turn unreliable.
Every field collected brings a load: it has to be filled, it has to be validated, it has to be kept current, and holding it creates a responsibility. That is why the field list stays short and grows as the need appears.
The last question matters. Fields such as identity, health or financial information get collected only where they are genuinely needed; where they are not, leaving them out may lighten both the responsibility and the upkeep.
A field dictionary is the list where every field you collect is defined in one place. Its purpose is to keep the same field from meaning two things to two people. If some write a full name and others only a first name into customer name, that field no longer measures one thing.
The dictionary is not a technical document. A table or a single page is enough. What matters is that each field is defined under six headings and that this definition is open to everyone who enters data.
For fields with a list of options, write the options too. Where free text is left, the same thing gets written ten different ways and the field becomes impossible to report on.
The one format rule can only be applied once the format is written. For the same spelling to be used at every point of entry, what that spelling is gets decided first and worked into the field dictionary along with an example.
Format decisions look small, yet most silent faults come out of them. Details such as the leading zero of a phone number, the order of day and month in a date, or a city written in capitals, break matching.
Leave the format to the form rather than to the user. A list of options, a date picker and a masked field can stop the difference at entry; it comes cheaper than cleaning up afterwards.
To apply the one row rule, a key that answers are these two records the same person is needed. The key is a field that separates the person from others and does not change over time.
A good key stands on its own. Where none can be found, two fields get combined. Name and surname alone do not make a key, because the same name can belong to several people and its spelling shifts.
Phone number, mail address, customer number, tax or identity number. Whichever is filled for every customer in your business is the candidate.
It has to be filled for every customer, be unique, stay unchanged over time and be written in one format. The field holding all four is the key.
When a second record arrives with the same key, the system updates the existing record instead of opening a new one and writes the touched fields into the history.
Decide here as well what happens when the same customer writes through two channels. If the channel is held in a separate field, the customer stays one row and the history does not split.
The cheapest place for cleanliness is the point of entry. Correcting a field after it went in wrong comes dearer than stopping it at entry. That is why validation moves into the form.
Validation works in two layers: format validation and meaning validation. Format probes that the spelling follows the rule; meaning probes that the value makes sense. With both in place, the records left to clean up later drop sharply.
If the same information sits in more than one place, which one counts as correct has to be written. Without that decision, every system updates its own copy and after a while which one is current stays unknown.
A single source does not mean the information sits in one place. It means it is updated in one place and flows from there to the others. Copies get read, not updated.
This decision fits into one table and sits beside the field dictionary. When it is unwritten, the argument gets had from scratch each time.
Data is not something cleaned once and left. People move, change numbers, company names change. That is why cleaning is not a project but a small and regular piece of upkeep.
You can have the automation do the upkeep itself. A simple check that lists mandatory fields left empty, warns when two records open under the same key, and marks values that break the format is enough in most businesses.
Collecting customer data is not only a technical decision. The customer wants to be informed about what is collected, what it will be used for and how long it will be held. This lesson gives no legal advice; its aim is to help you place the subject inside the process.
In practice three things make the work easier: the purpose of collection being written, where and how consent was taken sitting in the record, and the retention period being decided up front. All three can be kept beside the field dictionary.
The details can differ by country and by sector. Keeping the number of collected fields limited to those genuinely used may lighten the load under this heading as well.
Let us work the method through a single form from end to end. The example goes through the enquiry form of a service business.
Field selection: the business is thinking of putting seven fields on the form. For each field the question at which step will this be read gets asked. The date of birth and occupation fields turn out to be read in none of the flows and come off the list. What remains: name, phone, mail, district, enquiry subject and description.
Field dictionary: each field is defined under six headings. The enquiry subject stops being free text and turns into a list of options, because it has to be reported on.
Format: a single spelling is chosen for the phone and a masked field goes on the form. The district comes from a list of options. Name and surname are held in one field and corrected to capitalised initials.
Key: the phone number is chosen as the key, because it is filled on every enquiry and held in one format. When a second enquiry arrives from the same number, no new record opens and the new enquiry is added to the existing one.
Validation: the form cannot be sent while mandatory fields are empty. If the number is already on file, the existing customer is shown on screen.
Single source: one system is chosen as the source of customer information. The other screens read from it, and updating happens only at the source.
Upkeep: a simple check running once a week lists records with an empty mandatory field and pairs opened under the same key.
The return of this lesson comes from building your own field dictionary. It is short enough to finish in one sitting, and afterwards it makes every automation decision easier.
The list in the last item is the photograph of your data order today. Once the upkeep is set up, you can pull the same list again and compare.
Download the data inventory (xlsx) · which field, what it means, who enters it, where it lives and how long it is kept; format rules and deduplication are in it too.
An unused field does not get filled, and where it does, its accuracy goes unchecked. What remains is a field both empty and unreliable, while the responsibility for holding it carries on.
Every field left as free text scatters into different spellings over time. No report can be pulled and matching does not work.
Where no deduplication key is chosen, the same customer opens more than once. The history splits, messages repeat, and the customer notices.
When a field name changes, the flows tied to it quietly start reading an empty value. The change gets made in the dictionary first, then the flows get updated.
An update made on the copy instead of the source disappears at the next refresh. The user sees the change vanish and trust in the system falls.
Data cleaned once scatters again over time. Small and regular upkeep runs more easily than a large and postponed cleanup project.
Before opening a new form or a new field, you want to be able to say yes to the items below.
The terms that come up in data conversations. Once the team shares this language, field arguments get shorter.
In this lesson you built the field dictionary and decided where the data lives. What you hold now is a measurable task, a written process and defined fields. The next lesson deals with which tool it gets built with.
In the fourth lesson, off the shelf, no-code and custom builds get scored against seven criteria; the field dictionary from here becomes the ground of the data ownership criterion.
Scoring off the shelf, no-code and custom builds against seven criteria. With a downloadable comparison table.
Boundary, steps, decision points and exceptions. The fields in this lesson get marked on that map.
The whole training section and the modules added since.
We can review together whether the fields you collect are earning their place. Writing to us is enough.
Get a Quote on WhatsApp