How overlapping statements are handled
Why the same transaction can arrive twice, and what a careful importer does about it.
Anyone who imports more than one statement eventually hits the same problem. The January statement runs from the first to the thirty first. The February statement, for reasons that make sense to the bank and nobody else, starts on the twenty fifth of January. Import both and a week of transactions exists twice.
Why a simple check is not enough
The obvious fix is to compare date, description and amount, and drop anything that matches something already stored. It works until it does not. Two coffees at the same shop on the same day for the same price are two real transactions, not one transaction imported twice, and a blunt check silently deletes one of them.
The opposite failure is just as common. The same payment can appear with slightly different descriptions in two statements, because one document truncates the merchant reference and the other does not. A strict comparison sees two different transactions and imports both.
Signals worth using
- Date and amount: the strongest pair, but far from unique on their own.
- Merchant and description: compared after normalization, so reference numbers and store codes do not break the match.
- Account: the same amount on the same day in two different accounts is two transactions.
- Source document: a row already imported from this exact file is a re-import, not a duplicate.
- Occurrence: where a genuine repeat exists, the position within the day distinguishes the second coffee from the first.
- Statement period: when two documents cover overlapping dates, everything inside the overlap deserves closer inspection.
Overlap as a first class idea
Statement periods are worth extracting on their own, not just as a label. Once an importer knows that this document covers the twenty fifth of January to the twenty eighth of February, and that another document already covers up to the thirty first of January, it can treat the four day overlap differently from the rest of the file. Inside the overlap, a probable match is a duplicate. Outside it, the same evidence is much weaker.
Skipping should be visible
Whatever the logic, the user has to be able to see it. An import that reports seven duplicate or overlapping transactions skipped, with a list you can open and inspect, is trustworthy. An import that quietly drops rows is not, because the first time it makes a mistake you have no way to find out.
The same applies in reverse. If you decide a skipped row is genuinely a separate transaction, you should be able to keep it. Karchu shows the skipped rows before the import completes and lets you merge, ignore or keep each one.
The practical rule
Prefer flagging over deleting. A duplicate that reaches your ledger is annoying and easy to spot. A real transaction that never arrived because software decided it looked familiar is invisible, and it will quietly break your reconciliation months later.