Putting a connection between two systems does not on its own make the two hold the same record with the same meaning. In this lesson you fill in the integration map: which field moves in which direction, how the format converts, how the same record avoids splitting in two and who is left waiting when the connection drops.
In the fifth lesson you built the first flow and integrations were a single line on the setup sheet. This lesson opens that line. Two systems understanding each other is not the same as putting a connection between them; the agreement is in place once it is written down which field moves in which direction, how the format converts and who is left waiting when the connection drops.
The lesson does not walk through the connector screen of a particular tool. What it covers is the integration map: the record you fill in before building the connection and still hold afterwards. Once the map is in hand, the same questions come out answered no matter which tool the connection is built with.
Prerequisites: the field dictionary from the third lesson and the flow built in the fifth. Without the field dictionary the mapping table cannot be written, because the definitions of the fields to be matched are not there yet.
Integration means two systems holding the same record with the same meaning. Having a connection in place does not give that on its own; the record is genuinely shared once both sides read the same field with the same meaning.
In practice the distinction gets measured with one question: when you open a record side by side in both systems, do the two say the same thing? If the answer is no, the connection works but the integration is not in place. That state does not raise an error either, because each system looks consistent within itself.
Fields defined against each other, the direction chosen, the format converted and the same record not split in two. All of them decisions that can be written down.
Putting a connection between two systems and watching the data flow. Whether the flowing data is read correctly is a separate question and it does not answer itself.
Every time you stop while mapping a field and wonder what the other side counts it as, that is a missing row on the map. A missing row comes out at the first unusual record.
The map sits on a single page and each row describes one field pair. The six columns correspond to the six questions asked while the connection is built; when one column stays empty, the behaviour of that field is left to the tool.
Filling the map in before building the connection is a different job from filling it in afterwards. Filled in first, the gaps come out on paper; filled in later, whatever behaviour the tool produced gets recorded, and wrong behaviour enters the map looking correct.
The last two columns are usually filled in last although they deserve to be asked first. Empty values and conflicts are the two places where integrations produce silent faults.
In every field pair one side is where the record is born and the other is its mirror. Choosing the source is not a preference but an observation: the source is wherever the field first comes into being. The mirror carries the record rather than changing it.
A two way direction can be built, but it has a price: once both sides can write, the chance of a conflict appears, and if the conflict rule is not written down the last writer wins. Who writes last usually depends on timing, so the outcome becomes hard to foresee.
Where the field is first born. Corrections happen here too; a correction made in the mirror can be taken back at the next transfer.
The side that holds the record in order to display it and run its own work. Once mirror fields are treated as read only, arguments in the team get shorter.
A field both sides can write. It is built only if the conflict rule is written; without a rule the field gets turned into one direction.
Two systems hold the same information under different names. The mapping table puts those names side by side, but the real work is matching definitions rather than names. Two fields with the same name describing different things is a more common situation than two differently named fields describing the same thing.
The field dictionary from the third lesson earns its keep here: because the meaning of the field is written down, the mapping discussion runs over definitions. Without the dictionary, mapping turns into placing two screens side by side and wiring boxes that look similar.
Once the mapping table is complete, no field between the systems is left unspoken. An unspoken field is the field that passes through the transfer silently empty.
Two systems may write the same value differently. The order of a date, phone prefixes, the separator in an amount and the capitalisation of a name are the classic places where a transfer breaks quietly. That break raises no error, because the target system can interpret the incoming value by its own rules.
Who performs the conversion is answered on the map: on the source side, during the transfer, or on the target side. All three can be used, but when the choice is not written down the conversion gets done in two places and the value breaks by converting twice.
Order and time zone are two separate questions. The same day can fall on two different days; when month and day swap places the record still looks valid, so it goes unnoticed.
Country prefix, spaces and separators. If the deduplication key is the phone number, a format difference splits the same person across two records.
Decimal separator, thousands separator and currency. An amount that does not carry its unit falls back to the default while being interpreted in the target.
If a conversion rule fits on one line of the map, it is clear enough. A rule that does not fit on one line is usually two separate rules tangled together.
The outcome transfers produce most often is the same record splitting in two in the target. The reason is usually this: the system looks at a key to tell whether the incoming record has been written before, and the key is either not chosen or not held in the same shape on both sides.
Choosing the key was the subject of the third lesson; here the job is confirming that the key sits in the same shape in both systems. If the key is held with spaces in the source and without them in the target, the two records look different and a duplicate appears in the target.
A duplicate record is a silent fault. Reports grow their totals, the customer gets reached twice, and after a while which of the two records is current can no longer be told apart.
When the record moves matters as much as what moves. Instant transfer is the most visible option, but not every job calls for moving things instantly; where a delay is acceptable, batch transfer runs with fewer moving parts.
Choosing the pace is not a speed preference but a waiting preference: if the transfer is late, who waits and does that person notice the delay? A delay that goes unnoticed comes back not as a fault but as "the data is wrong".
The record moves as soon as it is created. When it is not written down what happens if the other side does not answer, the record can be left hanging.
Records are written to a queue and moved in order. At the moment of a drop the queue waits, and once the connection returns it carries on from where it stopped.
Everything moves together at set intervals. The plainest option where delay is accepted; the transfer window gets announced to the team.
The question most often skipped while a connection is built is this: what happens to the records piled up until today? There are two separate jobs here and mixing them leaves both half done. The first load is a one off move; the ongoing sync runs everything from then on. One finishes and the other begins, and their rules do not have to be the same.
Moving existing data is not needed in every case. Starting "from today onwards" is a valid choice too and it is enough for most pilot tasks; the old records carry on sitting in their own system. Once the decision to move is taken, the scope gets written: from which date onwards and records in which state, closed jobs included or not.
The first load is the moment that presses hardest on the deduplication key. A format difference that goes unnoticed in records arriving one by one produces duplicates in bulk during a bulk move, and the count can grow past what anyone would clean by hand.
That is why the first load is done with real data rather than test data, and it needs to be reversible: a narrow date range moves first, the result is compared across both systems, and if no duplicates come out the rest follows. Marking the transferred records is what makes it possible to pick a wrong load back out of the target.
Every connection drops from time to time. The lesson does not offer to remove that possibility; it offers to write down where the work stops when it happens. Left unwritten, a drop surfaces not as a fault but as a gap noticed days later.
The drop row of the map holds three things: the person waiting, the manual fallback and the rollback. With those three written down a drop can be managed as a delay; unwritten, a drop can quietly turn into lost data.
A half transferred record is harder to notice than one that never moved: a record does appear in the target, so the gap only comes out when someone uses it.
A connection is built with an account, and everything that account can see may open up to the other side. Using the most privileged account during setup makes the work quicker and leaves behind a door nobody narrows afterwards.
Whether the account belongs to the work rather than to a person is a separate matter. A connection built with a personal account stops quietly when that person leaves, and the reason for the stop gets looked for in the connection itself.
The access rule written in the third lesson gets tested a second time here: a field whose audience was written in the dictionary may be opening onto another system through the integration.
After the connection is built, testing does not end with moving one record and looking at the result. Done together, the three passes verify most of the rows on the map.
Testing is done with test data and the test records are recognisable by their names. Testing with live records leaves a trace in the target system that has to be cleaned up.
A record with the non mandatory fields left blank. It shows what an empty value becomes in the target: does it stay empty, does a default get written, is the record refused.
The typical record of the daily work. Used for opening the field mapping and the format conversion side by side and comparing them.
A long name, a differently formatted phone number, a second record under the same key. The pass that tests the deduplication and conflict rules.
The result of the three passes gets written next to the map. A row that has not been tested does not count as built.
In the earlier lessons the office picked a pilot task, wrote its process down, drew up its field dictionary, chose its tool and built the first flow. Now the listings system and the customer tracking sheet do not see each other and the same enquiry is typed into two places by hand.
The office starts by filling in the map and an argument comes up on the very first row: in which system is the phone number born? The enquiry arriving in the listings system carries a phone number and so does the tracking sheet; the two sit in different shapes.
The source is marked as the enquiry record of the listings system; the tracking sheet is accepted as the mirror. For the phone field the format conversion is done during the transfer and turned into a single shape. The deduplication key is chosen as the phone number, because the same person can write in from different listings.
The empty value row is written like this: if the phone is blank the record is not written to the tracking sheet, it drops into a pending list and the office coordinator goes through that list at the end of the day. On the conflict row the source side is held above the other; corrections made by hand in the tracking sheet go into the note field rather than the phone field.
The pace is chosen as batch transfer, because moving enquiries a few times a day rather than the moment they arrive is enough for the work. On the drop row the person waiting is written as the office coordinator, and the old way of typing records in is kept open as the manual fallback.
The tracking sheet already held records, so the first load decision is taken as well. Instead of moving the whole history the office takes recent and open enquiries; closed jobs stay out, because their counterpart in the tracking sheet is already closed. The load is tried first over a narrow date range and the row counts are compared across the two systems. Duplicates come out on the first try: the phone format has not been levelled yet. The conversion is corrected and the load is repeated, the transferred records are marked with a tag, and the tag is what lets a load that goes wrong be picked back out of the target in bulk.
In the three pass test the unusual pass catches one gap: when a second enquiry arrives from the same phone number, a new record opens in the tracking sheet. The key rule is corrected and the second enquiry goes in as a note under the same person.
This week put one single connection on the map. You do not have to build a new connection; drawing the map of a transfer that already runs does the same job and usually brings out a few missing rows.
Download the integration map (xlsx) · a single page map that has you fill in the field mapping, the direction, the format conversion, the first load and the drop behaviour; the three pass test is in it too.
The mistakes below come out of the map being left incomplete rather than out of the connection itself. What they share is this: all of them run without producing an error message.
Once both sides can write, the last writer wins. Corrections made by hand can be taken back at the next transfer and the reason gets looked for in the tool.
If the same phone number sits in two different shapes, two records appear in the target. A duplicate record breaks both the report and reaching the customer.
An empty field can fall back to a default in the target. A default value looks like a filled in value, so it cannot be told apart afterwards.
If nobody is defined as waiting when the transfer stops, the gap can come out days later with a question from a customer.
When the test record is picked from live data, a trace that has to be cleaned stays in the target, and the cleaning is usually done halfway.
A map written after the build records the behaviour the tool produced. Wrong behaviour gets documented as if it were correct by the same route.
If every item on this list can be answered before the connection goes live, the map counts as complete.
Terms that come up in integration conversations. Speaking the same language as the person building the connection makes filling in the map shorter.
In this lesson you got two systems talking: you mapped the fields, chose the direction, converted the format, decided whether existing data moves and wrote the drop behaviour. The lessons ahead take on keeping that arrangement going: managing exceptions, bringing the team into the process and measuring the result.
If you would like to carry on from the published lessons, the module on where to look when a transfer does not run as expected is next.
Silent faults, the fault sheet, the halving method and a lasting fix.
The setup sheet and the five parts. The integration line this lesson opens sits there.
The whole training section and the modules added since.
If you would like to fill in the map together before connecting two systems, write to us.
Get a Quote on WhatsApp