Label with an AI assistant¶

Labelling is the slow part of training a model. The Sight labelling skill lets an AI assistant do it: it looks at each photo, draws tight boxes around the things you name, checks every box by looking at it again, and sets aside photos too unclear to label. Then it packs a dataset Sight can train on.

It works with any assistant that can see images and run a small Python script: Claude, ChatGPT/Codex, GitHub Copilot and others. Your photos stay on your computer; the assistant reads them where they are.

Download the skill

Install it¶

Unzip it where your assistant looks for skills:

Any other assistant can be given SKILL.md as its instructions. The scripts need Python 3.9 or later and Pillow (pip install pillow).

Use it¶

Ask, for example:

Label the photos in site-photos/ for hard hats, using the Sight labelling skill.

The assistant copies the photos into a dataset folder and goes photo by photo. For each one it does three things:

• Box every hard hat, by its corners.
• Check each box by drawing it back onto the photo and zooming in to tighten it.
• Decide the photos with nothing in them ("none"), and set aside the ones it cannot tell, with the reason.

When every photo is decided, it checks the dataset and packs dataset.zip. Upload that at Models → Upload dataset & train, or import it at Label to look over the boxes first. Sight sets aside its own photos to measure the model on.

The rules it follows¶

• Tight boxes, around the object only. A hard hat's box is the hat, never the head under it.
• Every part. That includes parts in shadow or a different shade, such as a hat's rear brim.
• Never guess. Something unclear gets no box, and a photo too unclear to trust is set aside with the reason. One wrong box costs more than one photo left out.
• Only the classes you name, spelled the way you name them.
• Photos with nothing in them are kept, about 1 in 5, so the model learns what is not one.

The script refuses a box that cannot be right, for example one outside the photo or a few pixels wide. It also refuses to pack a dataset until every photo is decided and every class has boxes.

How good is it?¶

We tried it on four photos from our own hard-hat test set, whose boxes a person had checked, and compared the assistant's boxes with theirs:

• Every photo was right. No hat was missed and no false box was drawn, and the fuse box with no hat in it was left empty.
• Large hats: the boxes matched the checked ones almost exactly (overlap 0.94 and 0.96, where 1 is identical).
• A small hat about 50 pixels wide: the assistant's box left out the hat's darker rear brim (overlap 0.61). The skill now tells assistants to look for exactly that.

So look over its boxes on small and distant objects before you train, which the Label page makes quick. For a model you rely on, a person checking the boxes is still worth it.