Stop the suite failing on things that are not test failures

Two sources of random red on WSL, neither of them a test.

A launch that never happened. The Godot binary is a Windows executable
started across the interop boundary, and with six going at once that start
occasionally just fails: exit 127, an empty log, no engine banner, nothing
run. It was indistinguishable from a failure and moved between tests from
run to run. Anything that actually ran got as far as printing the banner, so
the two are easy to tell apart; a start that produced no banner is retried
up to three times and reported as NOSTART if it never comes up. Retries are
printed, never hidden - a test that needed one is saying the machine is
struggling even where it went on to pass.

The working tree as a run directory. WSL ran the suite in place, so six
Godot processes wrote .godot and created and deleted files under
PlanetSetups/ inside the live repo, alongside git and whatever else was
using it. Single tests failed at random under that - a delete that did not
take, a scene that would not instantiate - never the same one twice. WSL now
copies to %LOCALAPPDATA% first, the same isolation Termux already gets from
its own copy, on a native NTFS path so the Windows Godot still reads it at
full speed. Costs about 10s a run and leaves the repo untouched.

Four consecutive clean 124/124 runs since.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Loïc Wisniewski 2026-08-22 23:30:18 +02:00
commit 85c9bd9d8d
2 changed files with 75 additions and 21 deletions

View file

@ -13,12 +13,12 @@
# Termux the Godot 4.7 Linux arm64 build installed inside the
# proot-distro Ubuntu container, which is the only way to run
# a desktop Godot on the phone at all.
# WSL the Windows Godot build, run straight from /mnt/c. A Linux
# `godot` on PATH is used first if there is one.
# WSL the Windows Godot build off /mnt/c. A Linux `godot` on PATH
# is used first if there is one.
#
# The platform decides which Godot runs and whether the project is copied
# first; the worker count and tests/_run_inside.sh, which does the actual
# running, are shared.
# The platform decides which Godot runs and where the project is copied to.
# Everything else - the copy itself, the worker count, and
# tests/_run_inside.sh, which does the actual running - is shared.
#
# On Termux, three things make this fast, in descending order of how much
# they matter (measured, not guessed - see tests/_profile_startup.gd):
@ -29,9 +29,15 @@
# alone took 8.5s per test, against ~2s from a native path. Nearly every
# test loads Main.tscn, and none of that time is gameplay. The copy is
# ~1.4s for the whole suite and its .godot import cache is kept between
# runs. WSL skips the copy: a Windows Godot reads C: natively, and the
# only thing copying to ext4 would achieve is putting the project behind
# the \\wsl$ share instead.
# runs. WSL copies too, but to a native NTFS path rather than to ext4:
# a Windows Godot reads C: at full speed and would read \\wsl$ slowly, so
# the copy is for ISOLATION rather than for speed there. Six Godot
# processes churning the project directory is six processes writing
# .godot, creating and deleting files under PlanetSetups/, and doing it
# in a working tree that git, an open editor and the player are also
# using. Under that, single tests failed at random - a delete that did
# not take, a scene that would not instantiate - never the same one
# twice.
# 2. Tests run JOBS at a time. Each is its own Godot process - they call
# quit() and stamp on the GameState autoload, so they cannot share one -
# which makes the suite a pile of independent, single-core-bound jobs on
@ -103,14 +109,31 @@ _pick_jobs() {
}
JOBS="${JOBS:-$(_pick_jobs)}"
VERBOSE="${VERBOSE:-0}"
# The copy only buys anything on Termux (see 1. above), so on WSL the project
# directory IS the run directory whatever IN_PLACE says.
if [[ "$PLATFORM" == "wsl" ]]; then
IN_PLACE=1
else
IN_PLACE="${IN_PLACE:-0}"
fi
WORK_DIR="${WORK_DIR:-$HOME/.cache/spaceship-tests}"
IN_PLACE="${IN_PLACE:-0}"
# Where the copy lives. Termux wants its own native filesystem; WSL wants a
# WINDOWS one, because the thing reading it is a Windows binary and $HOME is
# ext4 behind the \\wsl$ share. %LOCALAPPDATA% is the natural home for it;
# if that cannot be read, fall back to a sibling of the project - still on
# the same native drive, still outside the repo.
_default_work_dir() {
if [[ "$PLATFORM" != "wsl" ]]; then
echo "$HOME/.cache/spaceship-tests"
return
fi
local win_local
win_local="$(cmd.exe /c 'echo %LOCALAPPDATA%' 2>/dev/null | tr -d '\r')"
if [[ -n "$win_local" && "$win_local" != *'%'* ]]; then
local unix_local
unix_local="$(wslpath -u "$win_local" 2>/dev/null)"
if [[ -n "$unix_local" ]]; then
echo "$unix_local/spaceship-tests"
return
fi
fi
echo "$(dirname "$PROJECT_DIR")/.spaceship-tests"
}
WORK_DIR="${WORK_DIR:-$(_default_work_dir)}"
if [[ -z "${GODOT_BIN:-}" ]]; then
if [[ "$PLATFORM" == "wsl" ]]; then