The 64,000,000 in the official sample was an amount nobody paid

For about 3 weeks I believed the figure 64,000,000 was real. It sat inside an XML file distributed as the official sample for the annual form. Open it and the elements of the form are all laid out in order, with what look like real amounts inside them. This, we read, is what an actual filing looks like.

Here is how deep it had gone. We keep a table of the specification, 1 line per field, saying what goes into each one. 17 lines of that table named this file as their source. The fixture we tested our output against was a copy of the same sample with the amounts swapped out. Which means the thing deciding whether our output was correct was this file.

I found it by accident. I wanted to check an encoding question, so I opened the file unformatted in a text editor. There was 1 line of comment at the top. It said: Created with Liquid Technologies. A little further down, the taxpayer number field was 16 x characters in a row. Reading on, every optional field was filled in too, and every string field was exactly the maximum length the schema allows.

So the file was a sample a machine had generated from the schema (the XSD). Element names, the order they come in, how they nest, which are mandatory and which optional — about the shape, it could not be more accurate. The values inside it, though, are what a generator put there to fill a field of that type. Nobody paid 64,000,000. No office ever received it.

Here is what happened. Because the sample had the optional sections filled in as well, we were emitting one of those sections every time, regardless of the condition it belongs to. The real Coretax does not accept a file carrying that section. When it was first rejected, we spent about 4 days looking inside our own code. Our output did not differ from the sample by a single character, so the cause had to be somewhere else on our side. That the premise itself might be wrong was not a thought we had.

I asked V. She is an accountant in Jakarta who has been testing KAZENA Books with the real data of her own practice. When I asked her whether an annual form actually filed looks like this, she opened one her office had filed and told me which elements were in it and which were not. The section in question had never been there at all. What 3 weeks of reading the sample had not told me, one real document answered in one go.

Here is the rule we settled on. Every line of the specification table carries its source, and the grade of that source. There are 4 grades. First, the real thing — a file the live system produced or accepted. Second, a person's confirmation — someone who does the work checked it on the live screen, with the date they checked written alongside. Third, the text of the regulation, or the schema itself. Fourth, a sample a machine generated, and our own guesses. And we decided that no behaviour resting only on the fourth grade goes into the product silently.

We re-sourced those 17 lines under the rule. 6 were replaced by the real thing. 5 were backed by the text of the regulation. 4 were filled in by V's confirmation. The remaining 2 had no source anywhere. That is: there was no reason for them beyond the sample saying so. The behaviour those 2 lines supported has been taken out of the product.

The word official was doing a great deal of work here. The file is distributed officially, so every byte inside it must be official too — that is how we read it. What is actually official is the schema the file conforms to. The numbers inside were decoration, placed there to explain the schema. There is nothing wrong with the sample. What was wrong was us, not reading a sample as a sample.

We fixed the product side as well. Before, when something was missing, we said only: this field is required. Now we separate out the reason. A field the schema marks as mandatory and a field that got a filing rejected when it was left out are 2 different sentences. Our test fixtures are no longer copies of the sample either; they are built from files the live system actually accepted. The numbers in them are replaced with something obviously invented — 16 nines in a row, for instance. We would rather not keep real numbers inside our tests.

If you are building something against a government system, there is 1 thing I would pass on. Check who produced the number you are relying on. A value a machine put into a schema, a value printed in documentation, a value a practitioner gave you from memory, and a file the system itself produced — on screen, these 4 wear the same face. The same black characters, the same number of digits. What differs is how far back you have to go to put things right when one of them turns out to be wrong.

For the avoidance of doubt: I am not a tax accountant. What goes into the annual form, and which fields are mandatory, is decided on the regulatory side and it changes. What is written here is a record of our own mishandling of evidence. Please confirm the actual filing decisions with a professional.

One last request. If a field in KAZENA says it is required and you cannot see why, please tell us. There is a chance our own grounds for it are still sitting at the fourth grade. The grade of the evidence is invisible to the person using the software. Keeping something invisible correct is our job.