Theia

The annotator

A browser tool to digitize landmarks on your photographs. The photographs stay on your computer: the page reads them from a folder and uploads nothing. When you are done, you export a landmark file.

• Write your landmark scheme, or click the landmarks in order on the first photograph.
• Once one photograph is finished, each new one opens with the landmarks proposed from the finished ones. You drag them into place.
• Zoom, pan and a magnifier for exact placement, keyboard shortcuts, undo and redo.
• Mark a landmark as missing, or place several specimens on one photograph.
• Your work is saved in the browser as you go.
• Opens and writes TPS, NTS, COCO keypoints, YOLO pose and the CSV of the Theia app.

From your photographs to a model pack

A web-based training pipeline turns annotated photographs into a taxon-specific model for the app, in four steps.

1. 1

   Conditioning

   Your photographs and landmarks are checked for duplicates, landmarks outside the frame and shapes far from the average, which often means two landmarks were swapped. Whole groups, such as populations, are held out for validation. You look at everything before training starts.

2. 2

   Training

   An object-detection model learns to locate the structure in the photograph. A pose-estimation model learns to predict your landmarks on it.

3. 3

   Validation

   The model predicts the landmarks on the held-out photographs. GPA, PCA and a Procrustes ANOVA run on your manual landmarks and on the model's, and the report says whether both lead to the same biological conclusions, as in the paper.

4. 4

   Model pack

   Both models, the landmark scheme and a self-test go into one file that you install in the Theia app. From then on, inference runs on-device and offline, in the field.