A self-checking payments data pipeline

RepairDesk earns revenue from payment processing as well as software. The processor reports each merchant's monthly volume and commission in a file keyed by its own merchant ID, which means nothing to our CRM.

To use the data, someone had to join that file to a separate list linking merchant IDs to stores, fix whatever didn't match, and import the result by hand. It was slow, easy to get wrong, and by the time I picked it up, seven months of files were waiting.

• The data is financial, so nothing could reach the CRM without a person approving it first.
• The merchant-to-store list changes over time and people correct it by hand. Those corrections couldn't be overwritten.
• Unmatched merchants are resolved by a team, not one person, so the system had to keep asking until someone did.
• The backlog had to be cleared without anyone tracking which month came next.

Six n8n workflows around a shared Google Sheet that acts as the control panel: a master link list, a review tab, an exceptions queue and an activity log.

• Intake. A simple upload form drops files into the right folders, so nobody needs to know the folder structure.
• Link list sync. Takes the latest merchant-to-store list, keeps manual corrections, and flags conflicts for a person instead of guessing.
• Monthly build. Finds the next unprocessed month and handles just that one. Running it again clears the next, which is how the backlog was worked through.
• Exceptions. Unmatched merchants go to a queue, and the team gets a reminder every day until each one is resolved or marked as skip.
• CRM push. Runs only on approved rows, checks for duplicates before writing, and handles each row on its own so one bad record never stops the batch.
• Error handler. A separate workflow that logs and alerts on any failure. Updates, reminders and failures each have their own chat channel so the important ones don't get buried.

Most of the work was in the failures. A few that taught me the most:

• CRM IDs changed on their ownA spreadsheet parser turned 19-digit IDs into rounded numbers, silently pointing records at the wrong customer. I replaced it with a plain-text parser that never converts IDs.
• Only 1 of 60 rows processedA step was set to run once for the whole batch instead of once per row. Fixing the mode solved it, and taught me to test with real batch sizes.
• "No match" crashed the runThe CRM returns an empty response when a search finds nothing, which broke parsing. The empty case is now an expected, handled result.
• The same exceptions, every monthUnresolved merchants were re-added each run, piling up seven or eight copies. The queue now checks what's already pending first.
• Renamed steps broke silentlyRenaming a step left references pointing at nothing. I wrote a script that checks every workflow for broken references before handover.

Processing data now sits next to each customer in the CRM, where Sales, Success and Finance already work. The monthly job went from a manual join to uploading a file and approving a review tab. Anything unusual is surfaced and chased automatically.