Complete CAE workflow
Can an agent complete a CAE workflow from CAD and a test specification—prepare geometry, generate a mesh, define loads and supports, solve, and validate the results?
- Evaluation
- Complete CAE workflow
- Runs
- Up to three per model
- Time limit
- 3 hours per rollout
- Resources
- 4 CPUs · 16 GiB RAM
Leaderboard
| # | Model / harness | Score ± SD | Overall pass |
|---|
Mean ± sample standard deviation across three independent rollouts. Dots show individual scores; the vertical marker shows the mean. Overall pass requires all five components and boundary conditions to pass.
Rollouts
Task & environment
Results describe performance on the selected task, rather than engineering ability across all domains. Scores are not pooled across evaluations. Three rollouts show repeatability but provide limited evidence for small differences between models.
Scoring
Passing components contribute their score × weight. Failed components contribute zero. Boundary-condition compliance is required for score credit.
Peak stress & location 35%
Score = 1 / (1 + max(stress error / 10%, location error / 10 mm)²). Both tolerances must pass; location uses the permitted symmetry.
Stress field 20%
Volume-weighted stress agreement across the spoke region, with at least 97.5% coverage and valid native mesh evidence.
Geometry 20%
Surface and volume agreement between submitted wheel CAD and the reference geometry.
Load & offset 20%
Prescribed inputs, arithmetic, applied moment, calculator tests and offset accuracy. Offset score = 1 / (1 + (absolute error / 2 mm)²).
Mesh efficiency 5%
min(successful distinct wheel meshes / 2, 1) / (1 + failed declared attempts). Synthetic probes are excluded.
Boundary conditions Required
Native solver evidence for a supported rim, a connected hub bending load and a deformable load path. This is a basic sanity check, not engineering certification.