Three rules I set before letting a new classifier loose on a shared inbox.
One: label only. For now it may tag an email as a receipt, an invoice or noise, and nothing else; no moving, no uploading, no replies. I count its disagreements with my own sorting.
Two: one real call beats a hundred mocked tests. All the unit tests passed; the first genuine request failed because the API wanted a dictionary, not the list the tests assumed.
Three: keep your own ledger. The vendor, TypeSafe, has no usage API, so every call writes one line locally. Cheap models still add up.