We turned what we checked on one form into a rule for five

On the morning of 23 September a message from V arrived, 1 line long. An employee is missing from the file exported for the monthly filing. She is an accountant in Jakarta who tests KAZENA Books with her own firm's real data. No error. No warning. It was simply that the people on her firm's list and the contents of the file did not match.

There was nothing to investigate about whose fault it was. A rule we had added 2 weeks earlier was leaving that person out of the output. It was not a defect. A rule we wrote on purpose was taking effect somewhere we had not intended.

Here is how it happened. On 8 September we sent the annual form to the real Coretax and 1 record was rejected, because the taxpayer number field had been left empty. A record with no number is not accepted on the annual form. That is not a guess; we confirmed it in the error the tax office's own screen returned.

As a fix, we excluded records with no number from the scope of the export. For the annual form I still think that judgement is right. Sending something that will be rejected anyway only spends the user's time.

The problem was where we put the rule. We put it in the shared part of the export. KAZENA Books can export 5 forms, and every one of them goes down the same path: gather the records in scope, then assemble the file. The rule that drops records with no number went into that gathering step. A rule we had checked once took effect on all 5.

The annual path has a check screen before the export. A record dropped from scope appears there as an item to fix. On the annual form, V would have noticed. The monthly path has no such screen. Gather, build, download. That is all. A person could go missing and nothing appeared on the screen.

I asked V what the monthly forms do. On the monthly forms, a person with no number can still be reported, because the form itself provides a way of writing the case where there is no number. In other words the tax office will accept them. We were quietly removing people the tax office would have accepted.

This is the heaviest part of it. The observation itself was real. The rejection on 8 September actually happened. What was wrong was how far we extended it. We took what we saw on 1 form and wrote it as a rule that did not care which form. The range we had checked and the range the rule reached matched only inside the head of the person who wrote it.

This way of being wrong feels worse to me than a rule written on a hunch, because you believe you have grounds. You can say you confirmed it against the real system. You can say it, but only about 1 form; about the other 4 you confirmed nothing. Unless you write down what you checked separately from how far that check reaches, even the person who wrote it loses the distinction.

Here is the fix. A rule now carries which forms it applies to. This rule applies to the annual form; it was checked on 8 September; the grounds are the error the tax office returned. Those 3 things are written beside the rule. It does not apply to forms that are not written there. We made the default not to apply. Letting through what we have not checked does less damage than stopping it.

We changed 1 more thing. After an export, we reconcile the counts: the number of records in scope and the number that made it into the file. If they differ, we print 1 line per record for the difference. With that in place, a rule reaching too far like this one would have been caught on our side. To notice that something has disappeared, you need to keep the count from before it disappeared. We had not been doing something that obvious.

V's messages are always short. This one was 1 line again. But what that 1 line pointed at was one person who did not make it into the output. Software talks in detail about what it produced and says nothing about what it did not. A build that cannot say why it left something out looks the same when it is working and when it is wrong.

One thing to be clear about. I am not a tax accountant. Which items are required on which form, and how a person with no number is reported, are decided on the regulatory side and they change. What I have written here is a record of our getting the reach of a rule wrong. Please confirm the actual filing with a professional.

A request to close with. If a file KAZENA exports ever fails to match the count on the list in your hand, tell us just that much. We will find the reason on our side.