AgentModelAgent vs single-shot AI CAD: drawings and mechanical parts
Can a model that checks its own work beat a fast model that answers in one shot? We gave the same six requests to three single-shot models inside AgentModel and to Claude Opus 5.5 running as an agent. The agent writes a script for SolidCraft, a B-rep parametric CAD tool, renders the result, measures it and fixes it until it matches.
Three of the requests are dimensioned drawings: a flange, an L-bracket and a spacer block. The other three are written specs: a split tube clamp, a pillow-block bearing housing and a knurled control knob.
Key findings
- Every arm built every part it was given. Mean judge scores were close: Opus 5.5 agent 3.83, both Gemini Flash models 3.50, and GPT-6 Astra 4.00 on the three parts it ran.
- The agent's lead comes from the written specs: it was the only arm to get the bearing block's slotted mounting holes and the clamp's flat bolted flanges (4 vs 3 on both).
- From drawings, every model was accurate: flange and L-bracket scored 4–5 across the board. Every arm, the agent included, drew the spacer block's slot with angled walls.
- The agent is much slower: its median session ran 152 s against 10–15 s for one Flash generation. The knurled knob took hours because SolidCraft booleans kept failing on dense knurl cuts.
Scoreboard
| Contender | Mode | Built | Mean score | Median time | Cost / tokens |
|---|---|---|---|---|---|
| Gemini 3.7 Flash · OpenSCAD | Single shot | 6/6 | 3.50 | 10 s | $0.0672 |
| Gemini 3.8 Flash · OpenSCAD | Single shot | 6/6 | 3.50 | 15 s | $0.20 |
| GPT-6 Astra + Pro brief · OpenSCAD | Single shot | 3/3 | 4.00 | 11 s | $0.20 |
| Opus 5.5 agent · SolidCraft | Agent (self-checking) | 6/6 | 3.83 | 2.5 min | 615k tokens |
- Gemini 3.7 Flash · OpenSCAD
- Gemini 3.8 Flash · OpenSCAD
- GPT-6 Astra + Pro brief · OpenSCAD
- Opus 5.5 agent · SolidCraft
Score by part
| Part | Gemini 3.7 Flash OpenSCAD | Gemini 3.8 Flash OpenSCAD | GPT-6 Astra + Pro brief OpenSCAD | Opus 5.5 agent SolidCraft |
|---|---|---|---|---|
| Round flange from a drawing | 4 | 4 | 4 | 4 |
| L-bracket from a drawing | 4 | 5 | 4 | 4 |
| Slotted spacer block from a drawing | 3 | 3 | not run | 3 |
| Split tube clamp for 25 mm tubing | 3 | 3 | not run | 4 |
| Pillow block for a 608 bearing | 3 | 3 | not run | 4 |
| Knurled thumb knob for an M6 nut | 4 | 3 | 4 | 4 |
Ringed scores are the best on that part. Click a part to jump to its renders.
Results
#Round flange from a drawing
Model the part shown in this mechanical drawing exactly as dimensioned. All dimensions are millimeters. Expose the diameters, thickness, hole count and PCD as parameters.
Expected size: 80 × 80 × 6 mm
Gemini 3.7 Flash OpenSCAD4
4 views ↗
- Judge
- fidelity 4 · printability 4
- Size
- 80 × 80 × 6 mm (0.0% off spec)
- Time
- 4 s · $0.0057 · 4k tokens
The generated model accurately reflects the 2D drawing, including the outer diameter, thickness, central bore, and the four bolt holes on the correct pitch circle diameter.
Gemini 3.8 Flash OpenSCAD4
4 views ↗
- Judge
- fidelity 4 · printability 4
- Size
- 80 × 80 × 6 mm (0.0% off spec)
- Time
- 9 s · $0.0075 · 5k tokens
Model accurately reproduces the flange geometry with correct proportions for the bore and bolt holes. Highly printable flat orientation.
GPT-6 Astra + Pro brief OpenSCAD4
4 views ↗
- Judge
- fidelity 4 · printability 4
- Size
- 80 × 80 × 6 mm (0.0% off spec)
- Time
- 8 s · $0.0571 · 5k tokens
Accurate model matching all features and proportions of the reference drawing. Fully printable.
Opus 5.5 agent SolidCraft4
4 views ↗
- Judge
- fidelity 4 · printability 4
- Size
- 80 × 80 × 6 mm (0.0% off spec)
- Time
- 44 s · 78k tokens
Accurate reproduction of the flange. All features including the central bore and bolt hole pattern are present and correctly proportioned. Easily printable.
#L-bracket from a drawing
Build this L-bracket from the attached drawing. Dimensions are in millimeters; keep the hole positions exactly where drawn and make plate thickness and hole diameter parameters.
Expected size: 60 × 40 × 40 mm
Gemini 3.7 Flash OpenSCAD4
4 views ↗
- Judge
- fidelity 4 · printability 4
- Size
- 60 × 40 × 40 mm (0.0% off spec)
- Time
- 10 s · $0.0088 · 5k tokens
Accurate L-bracket matching all dimensions and hole placements. Good proportions and fully printable as oriented.
Gemini 3.8 Flash OpenSCAD5
4 views ↗
- Judge
- fidelity 5 · printability 5
- Size
- 60 × 40 × 40 mm (0.0% off spec)
- Time
- 15 s · $0.0137 · 6k tokens
Model perfectly matches the requested geometry, dimensions, and hole placements. Fully printable as oriented.
GPT-6 Astra + Pro brief OpenSCAD4
4 views ↗
- Judge
- fidelity 4 · printability 4
- Size
- 60 × 40 × 40 mm (0.0% off spec)
- Time
- 11 s · $0.0616 · 6k tokens
The model accurately reproduces the L-bracket with correct proportions and hole placements. It is fully printable as oriented with the base flat on the build plate.
Opus 5.5 agent SolidCraft4
4 views ↗
- Judge
- fidelity 4 · printability 4
- Size
- 60 × 40 × 40 mm (0.0% off spec)
- Time
- 50 s · 77k tokens
Accurate L-bracket matching all drawing dimensions and hole placements. Easily printable as oriented.
#Slotted spacer block from a drawing
Create the slotted block in this drawing. Millimeters. The slot runs the full length; the two holes go all the way through.
Expected size: 50 × 30 × 15 mm
Gemini 3.7 Flash OpenSCAD3
4 views ↗
- Judge
- fidelity 3 · printability 4
- Size
- 50 × 30 × 15 mm (0.0% off spec)
- Time
- 7 s · $0.0075 · 5k tokens
Overall dimensions and holes appear correct, but the slot incorrectly features angled walls instead of straight vertical ones, and there are minor geometric artifacts at the ends of the slot.
Gemini 3.8 Flash OpenSCAD3
4 views ↗
- Judge
- fidelity 3 · printability 4
- Size
- 50 × 30 × 15 mm (0.0% off spec)
- Time
- 10 s · $0.0096 · 5k tokens
Overall dimensions and holes appear correct, but the slot incorrectly features angled walls instead of straight vertical ones, and there are minor geometric artifacts at the ends of the slot.
GPT-6 Astra + Pro brief OpenSCAD
Not run on this part.
Opus 5.5 agent SolidCraft3
4 views ↗
- Judge
- fidelity 3 · printability 4
- Size
- 50 × 30 × 15 mm (0.0% off spec)
- Time
- 55 s · 80k tokens
Block includes the slot and holes, but the slot has angled walls instead of being rectangular, and there are unexpected notches on the ends.
#Split tube clamp for 25 mm tubing
Two-piece split ring tube clamp for a 25 mm OD tube: 4 mm wall, 12 mm wide, two flanges each side with Ø4.5 mm holes for M4 bolts 8 mm from the tube surface, and a flat mounting foot on one half with two Ø4.5 slots. Show both halves side by side.
Gemini 3.7 Flash OpenSCAD3
4 views ↗
- Judge
- fidelity 3 · printability 4
- Size
- 125 × 37 × 12 mm
- Time
- 22 s · $0.0217 · 8k tokens
Two halves and mounting foot present. Connecting flanges are malformed as open hooks instead of flat flanges with bolt holes.
Gemini 3.8 Flash OpenSCAD3
4 views ↗
- Judge
- fidelity 3 · printability 3
- Size
- 138 × 12 × 37 mm
- Time
- 64 s · $0.0459 · 14k tokens
Model includes both halves with requested features. The mounting half has slots, but its raised flanges create overhangs.
GPT-6 Astra + Pro brief OpenSCAD
Not run on this part.
Opus 5.5 agent SolidCraft4
4 views ↗
- Judge
- fidelity 4 · printability 4
- Size
- 80 × 46.5 × 12 mm
- Time
- 2.7 min · 84k tokens
Model includes all requested features like flanges, holes, and slotted mounting foot. Good flat orientation.
#Pillow block for a 608 bearing
A pillow block housing for a 608 bearing (22 mm OD, 7 mm wide): press-fit bore with 0.1 mm interference, 6 mm wall around the bearing, a flat base 50 × 20 mm with two Ø5.5 mm slotted mounting holes, and a small set-screw hole from the top.
Gemini 3.7 Flash OpenSCAD3
4 views ↗
- Judge
- fidelity 3 · printability 4
- Size
- 50 × 20 × 37 mm
- Time
- 15 s · $0.0134 · 6k tokens
Matches basic shape and features. Mounting holes are round, not slotted. Proportions seem slightly off.
Gemini 3.8 Flash OpenSCAD3
4 views ↗
- Judge
- fidelity 3 · printability 4
- Size
- 50 × 20 × 39 mm
- Time
- 2.4 min · $0.11 · 32k tokens
Recognizable pillow block but misses slotted holes. Base proportions are incorrect, appearing narrower than the bearing housing.
GPT-6 Astra + Pro brief OpenSCAD
Not run on this part.
Opus 5.5 agent SolidCraft4
4 views ↗
- Judge
- fidelity 4 · printability 4
- Size
- 50 × 20 × 37 mm
- Time
- 2.5 min · 85k tokens
Accurately generates the pillow block with all requested features including slots, bore, and set-screw. Clean geometry and easily printable.
#Knurled thumb knob for an M6 nut
A knurled thumb knob, 30 mm diameter and 14 mm tall, with a diamond knurl on the outside, a hex pocket underneath for an M6 nut (10 mm across flats, 5 mm deep) and a Ø6.5 mm through hole. Filleted top edge.
Expected size: 30 × 30 × 14 mm
Gemini 3.7 Flash OpenSCAD4
4 views ↗
- Judge
- fidelity 4 · printability 4
- Size
- 30 × 30 × 14 mm (0.0% off spec)
- Time
- 10 s · $0.0100 · 5k tokens
Knob features diamond knurl, through hole, and fillets. Hex pocket visibility is blocked by render angles. Overall geometry matches well.
Gemini 3.8 Flash OpenSCAD3
4 views ↗
- Judge
- fidelity 3 · printability 4
- Size
- 30 × 30 × 14 mm (0.0% off spec)
- Time
- 15 s · $0.0125 · 5k tokens
Knurled knob with through hole generated. Knurl is aggressive. Top fillet is missing. Hex pocket cannot be verified from these views.
GPT-6 Astra + Pro brief OpenSCAD4
4 views ↗
- Judge
- fidelity 4 · printability 4
- Size
- 30 × 30 × 14 mm (0.0% off spec)
- Time
- 25 s · $0.0824 · 5k tokens
Visible geometry matches the prompt well, featuring diamond knurling, a through hole, and a filleted top edge. Bottom hex pocket is unverified.
Opus 5.5 agent SolidCraft4
4 views ↗
- Judge
- fidelity 4 · printability 4
- Size
- 30 × 30 × 14 mm (0.0% off spec)
- Time
- 3.6 h · 211k tokens
Visible features match well, including diamond knurl, fillet, and through hole. Hex pocket underneath is unverified. Good printability.
How this was run
Single-shot arms are one generation in the AgentModel app (OpenSCAD output) with no retries. GPT-6 Astra also received a build brief written by Gemini 3.1 Pro, and ran on three of the six parts.
The agent arm is Claude Opus 5.5 with a shell, the SolidCraft CLI and a written brief. It may render, measure and revise as often as it likes. Its time is the whole session, not one generation.
Every result is exported to STL and rendered the same way (four OpenSCAD views, perspective camera). Gemini 3.1 Pro scores each render 1–5 for overall match, fidelity to the request and printability, without knowing which arm made it.
Six parts is a small sample: read the per-part results, not just the averages.
Every model file is linked under its render. Open a .scad file in OpenSCAD or a .step file in any CAD tool to inspect the geometry yourself.