test: check what a real terminal renders for multi, zellij, and the TUI
tests/16-render.sh runs tmux, zellij, and the TUI in xterm on Xvfb, takes screenshots, and reads them back with tesseract: remote prompts in all panes, borders and titles with the hosts, [SYNC] appearing and disappearing, panes side by side, broadcast output in every pane and input in one pane only without it, windows split by panes_per_window, the status bar, and TUI marks. Screenshots stay in testenv/.artifacts. OCR reads three preprocessing variants (plain, stretched for dim gray, brightest channel for colored text) and normalizes characters that DejaVu Sans Mono makes ambiguous (1/l, C/(). The screenshots showed tmux cutting the session name in the status bar to "[netssh-pv"; netssh multi now widens status-left-length for its own session. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Sebastian UnterschützandClaude Opus 5.5 committed
1 parent
a31aab4110
commit
699b9469b6
12 files changed
+194
-5
No files matched your search
@@ -1,3 +1,4 @@
|
||||
.git
|
||||
testenv/.keys
|
||||
testenv/.state
|
||||
testenv/.artifacts
|
||||
@@ -23,3 +23,4 @@ Thumbs.db
|
||||
# local test environment (testenv/up.sh)
|
||||
testenv/.keys/
|
||||
testenv/.state/
|
||||
testenv/.artifacts/
|
||||
+2
-1
@@ -6,8 +6,9 @@
|
||||
- Fix `netssh sftp` and `netssh scp` with instance-qualified hosts (`prod/web01:/path`), which were taken for local paths.
|
||||
- Fix shell aliases of instance-qualified hosts ignoring anchored rewrite rules (`fsn_fsn1-pve1` instead of `fsn_fsnpve1`).
|
||||
- Add `netssh multi` to open several hosts side by side in tmux or zellij: host lists, `tag:`, NetBox virtualization clusters (`cluster:`, devices by default, `--kind vm|all`), and groups from the config (`@name`). `--sync` sends typing to all panes; `netssh sync` toggles it later. In the TUI, `Space` marks hosts and `Enter` opens them together. Configurable in `netssh configure` (multiplexer, panes per window, groups).
|
||||
- `netssh multi` widens the status bar of its own tmux session, which tmux otherwise cuts to `[netssh-pv`.
|
||||
- Cache entries remember the NetBox cluster of a host, for `cluster:` completion and offline use.
|
||||
- Add `testenv/`, a local Docker environment with two NetBox instances, SSH targets, a jump host, tmux, zellij, and bash/zsh/fish, and 160 end-to-end checks of all features, including `netssh multi` with real tmux and zellij (`testenv/up.sh`, `testenv/test.sh`). Not part of CI/CD.
|
||||
- Add `testenv/`, a local Docker environment with two NetBox instances, SSH targets, a jump host, tmux, zellij, and bash/zsh/fish, and 178 end-to-end checks of all features, including `netssh multi` with real tmux and zellij and screenshot checks of what a real terminal (xterm) renders (`testenv/up.sh`, `testenv/test.sh`). Not part of CI/CD.
|
||||
|
||||
## v0.0.23
|
||||
|
||||
|
||||
@@ -1085,6 +1085,7 @@ Both are non-blocking — the refresh runs as a background process and never del
|
||||
| Aliases missing in a new shell | `netssh hook install`, then a new terminal. The cache must not be empty. |
|
||||
| A host is named `sftp`, `scp`, `tunnel`, `hosts`, `multi`, `sync`, or like another subcommand | Use `netssh -- sftp`. |
|
||||
| `netssh multi needs tmux or zellij` | Install one of them (`apt install tmux`, `brew install tmux`). |
|
||||
| zellij shows empty boxes in its tab and status bar | Those are Powerline arrows; use a Nerd Font in your terminal, or add `simplified_ui true` to `~/.config/zellij/config.kdl`. |
|
||||
| `cluster:x matches no hosts` | Clusters select devices by default; VMs need `--kind vm` or `--kind all`. The name is matched case-insensitively. |
|
||||
| `netssh multi` panes use another config | Inside an existing tmux/zellij session, panes get your `HOME` and `XDG_*` paths, but not `NETSSH_*` variables (they would be visible to other users in the process list). Put such settings in the config file. |
|
||||
|
||||
@@ -1121,7 +1122,7 @@ End-to-end tests run against real services in a local Docker environment — two
|
||||
|
||||
```sh
|
||||
testenv/up.sh # build and start (first start ~6 min)
|
||||
testenv/test.sh # 160 checks: connect, strategies, instances, shortcuts, rules, sftp/scp, tunnels, notes, shells, TUI, configurator, multi with tmux and zellij
|
||||
testenv/test.sh # 178 checks: connect, strategies, instances, shortcuts, rules, sftp/scp, tunnels, notes, shells, TUI, configurator, multi with tmux and zellij, and what xterm renders (screenshots + OCR)
|
||||
testenv/down.sh
|
||||
```
|
||||
|
||||
|
||||
@@ -133,6 +133,7 @@ func TestTmuxOutsideSplitsWindowsAndAvoidsSessionNames(t *testing.T) {
|
||||
"tmux has-session -t =netssh-pve-2",
|
||||
"tmux new-session -d -x 240 -y 70 -s netssh-pve-2 -P -F #{window_id} #{pane_id} -n pve-1 ",
|
||||
"tmux new-window -t netssh-pve-2: -P -F #{window_id} #{pane_id} -n pve-2 ",
|
||||
"tmux set-option -t netssh-pve-2 status-left-length 40",
|
||||
} {
|
||||
if !strings.Contains(joined, want) {
|
||||
t.Errorf("missing %q in\n%s", want, joined)
|
||||
|
||||
@@ -42,6 +42,13 @@ func openTmux(panes []Pane, opts Options, r Runner) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !inside && i == 0 {
|
||||
// tmux cuts the status bar's session name at 10 characters
|
||||
// ("[netssh-pv"); widen it for this session only.
|
||||
if _, err := r.Output("tmux", "set-option", "-t", session, "status-left-length", "40"); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
window, pane, _ := strings.Cut(out, " ")
|
||||
if first == "" {
|
||||
first = window
|
||||
|
||||
+23
-1
@@ -57,6 +57,7 @@ public 10.231.0.0/24 + fd00:30::/64 mgmt 10.231.1.0/24 prod 10.231.2
|
||||
| `tests/NN-*.sh` | Test files; each runs in a fresh `HOME`. |
|
||||
| `tests/drive.py` | Drives `netssh configure` in huh's accessible mode. |
|
||||
| `.keys/`, `.state/` | Generated test key and seed output (git-ignored). |
|
||||
| `.artifacts/` | Screenshots from `tests/16-render.sh` (git-ignored). |
|
||||
|
||||
## Writing tests
|
||||
|
||||
@@ -82,9 +83,30 @@ reads the screen with `tmux capture-pane`. The multi test (`15-multi.sh`)
|
||||
opens panes in real tmux and zellij sessions and checks that broadcast typing
|
||||
reaches every node.
|
||||
|
||||
## What a user sees: `16-render.sh`
|
||||
|
||||
The render test runs tmux, zellij, and the TUI in a real terminal emulator —
|
||||
xterm on a virtual X server (Xvfb) — takes screenshots, and reads them back
|
||||
with OCR (tesseract). It checks what is actually drawn: remote prompts in all
|
||||
panes, borders and titles with the host names, `[SYNC]` appearing and
|
||||
disappearing, panes side by side, broadcast output in every pane, input in one
|
||||
pane only without broadcast, the status bar, and the TUI marks.
|
||||
|
||||
```sh
|
||||
testenv/test.sh render # then look at the screenshots:
|
||||
ls testenv/.artifacts/ # tmux-1.png, zellij-2.png, tui-3.png, …
|
||||
```
|
||||
|
||||
Helpers in `lib.sh`: `start_screen`/`stop_screen`, `term CMD…` (full-screen
|
||||
xterm), `shot NAME`, `screen_shows NAME TEXT…`, `screen_count NAME TEXT`.
|
||||
OCR reads each screenshot in three preprocessing variants (plain, stretched for
|
||||
dim gray, brightest channel for colored text) and normalizes characters it
|
||||
confuses in DejaVu Sans Mono (`1`/`l`, `C`/`(`), so checks stay robust without
|
||||
becoming vague.
|
||||
|
||||
## Not covered
|
||||
|
||||
- The installer (`install.sh`) downloads release assets from Gitea; it is tested by the Go tests with local fixtures.
|
||||
- macOS paths and behaviour.
|
||||
- `netssh uninstall` runs only with `--dry-run`, so the client keeps its binary.
|
||||
- tmux and zellij run headless inside the client; the checks read their state (`tmux list-panes`, `zellij action dump-layout`) and screen contents, not what a real terminal would render.
|
||||
- Rendering is checked in xterm with DejaVu Sans Mono. Other terminals and fonts may differ, e.g. zellij's Powerline arrows need a Nerd Font.
|
||||
@@ -11,8 +11,11 @@ COPY internal ./internal
|
||||
RUN CGO_ENABLED=0 go build -o /out/netssh ./cmd/netssh
|
||||
|
||||
FROM alpine:3.23
|
||||
# xvfb/xterm/imagemagick/tesseract: render panes in a real terminal emulator,
|
||||
# take screenshots, and read them back with OCR (tests/16-render.sh).
|
||||
RUN apk add --no-cache openssh-client-default tmux zellij bash zsh fish bash-completion \
|
||||
curl python3 jq ncurses coreutils \
|
||||
xvfb xterm imagemagick tesseract-ocr tesseract-ocr-data-eng font-dejavu \
|
||||
&& adduser -D -s /bin/bash tester
|
||||
COPY --from=build /out/netssh /usr/local/bin/netssh
|
||||
COPY testenv/client-entrypoint.sh /entrypoint.sh
|
||||
|
||||
@@ -165,6 +165,7 @@ services:
|
||||
- ./.keys:/keys:ro
|
||||
- ./.state:/state
|
||||
- ./tests:/tests:ro
|
||||
- ./.artifacts:/artifacts
|
||||
networks:
|
||||
public: {ipv4_address: 10.231.0.10, ipv6_address: "fd00:30::10"}
|
||||
mgmt: {ipv4_address: 10.231.1.10}
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
# What a user sees: tmux, zellij, and the TUI rendered by a real terminal
|
||||
# emulator (xterm on Xvfb), checked by OCR of screenshots. The screenshots stay
|
||||
# in testenv/.artifacts/ for a look by eye.
|
||||
config <<'EOF'
|
||||
resolver:
|
||||
strategies: [management_subnet, primary_ip]
|
||||
management_subnets: [10.231.1.0/24]
|
||||
ssh: {default_user: ops}
|
||||
groups:
|
||||
pve: ["cluster:pve-fsn1"]
|
||||
EOF
|
||||
check "refresh" netssh cache refresh
|
||||
export TMUX_TMPDIR="$HOME/tmux"
|
||||
mkdir -p "$TMUX_TMPDIR" "$HOME/.config/zellij"
|
||||
printf 'show_startup_tips false\nshow_release_notes false\n' >"$HOME/.config/zellij/config.kdl"
|
||||
start_screen && ok "X server and xterm" || bad "X server and xterm" "Xvfb did not start"
|
||||
|
||||
p1=fsn/fsn1-pve1.hoster.test p2=fsn/fsn1-pve2.hoster.test p3=fsn/fsn1-pve3.hoster.test
|
||||
prompts() { screen_shows "$1" "fsn1-pve1:~\$" "fsn1-pve2:~\$" "fsn1-pve3:~\$"; }
|
||||
# Two hosts on one OCR line means their panes sit side by side (tiled grid).
|
||||
side_by_side() { screen_text "$1" | ocr_norm | grep -F "$(printf %s "$2" | ocr_norm)" | grep -qF "$(printf %s "$3" | ocr_norm)"; }
|
||||
visible() { # NAME SHOT TEXT... — wait until the screen shows every TEXT
|
||||
local name=$1 shot=$2
|
||||
shift 2
|
||||
if wait_for 30 screen_shows "$shot" "$@"; then ok "$name"; else bad "$name" "$(screen_text "$shot")"; fi
|
||||
}
|
||||
|
||||
# --- tmux, started from a plain terminal: new session, attached in xterm ---
|
||||
term netssh multi --sync @pve
|
||||
visible "tmux: three remote shells on screen" tmux-1 "fsn1-pve1:~\$" "fsn1-pve2:~\$" "fsn1-pve3:~\$"
|
||||
visible "tmux: borders show host and [SYNC]" tmux-1 "$p1 [SYNC]" "$p2 [SYNC]" "$p3 [SYNC]"
|
||||
side_by_side tmux-1 "$p1" "$p2" && ok "tmux: tiled, first two panes side by side" || bad "tmux: tiled, first two panes side by side" "$(screen_text tmux-1)"
|
||||
tmux send-keys -t netssh-pve "echo typed-once-\$(hostname)" Enter
|
||||
visible "tmux: broadcast output in every pane" tmux-2 "typed-once-fsn1-pve1" "typed-once-fsn1-pve2" "typed-once-fsn1-pve3"
|
||||
tmux set-window-option -t netssh-pve synchronize-panes off
|
||||
tmux send-keys -t netssh-pve "echo just-this-pane" Enter
|
||||
sleep 1
|
||||
shot tmux-3
|
||||
[ "$(screen_count tmux-3 "just-this-pane")" = 2 ] && ok "tmux: without broadcast only one pane gets input" || bad "tmux: without broadcast only one pane gets input" "$(screen_text tmux-3)"
|
||||
shows_sync() { screen_shows tmux-3 "[SYNC]"; }
|
||||
shows_sync && bad "tmux: [SYNC] disappears when broadcast is off" "$(screen_text tmux-3)" || ok "tmux: [SYNC] disappears when broadcast is off"
|
||||
tmux kill-server
|
||||
|
||||
# --- more hosts than panes_per_window: further windows ---
|
||||
sed -i 's/^groups:/multi: {panes_per_window: 2}\ngroups:/' "$XDG_CONFIG_HOME/netssh.yaml"
|
||||
term netssh multi @pve
|
||||
visible "tmux: hosts split over windows pve-1 and pve-2" tmux-windows "pve-1" "pve-2"
|
||||
visible "tmux: status bar shows the whole session name" tmux-windows "[netssh-pve]"
|
||||
tmux kill-server
|
||||
sed -i '/^multi: {panes_per_window: 2}$/d' "$XDG_CONFIG_HOME/netssh.yaml"
|
||||
|
||||
# --- zellij: generated grid layout, broadcast through the sync tab ---
|
||||
term netssh multi --zellij @pve
|
||||
visible "zellij: three remote shells on screen" zellij-1 "fsn1-pve1:~\$" "fsn1-pve2:~\$" "fsn1-pve3:~\$"
|
||||
visible "zellij: pane titles show the hosts" zellij-1 "$p1" "$p2" "$p3"
|
||||
side_by_side zellij-1 "$p1" "$p2" && ok "zellij: grid, first two panes side by side" || bad "zellij: grid, first two panes side by side" "$(screen_text zellij-1)"
|
||||
session=$(zellij list-sessions --short --no-formatting 2>/dev/null | head -1)
|
||||
zellij --session "$session" action toggle-active-sync-tab
|
||||
zellij --session "$session" action write-chars 'echo zellij-$(hostname)'
|
||||
zellij --session "$session" action write 13
|
||||
visible "zellij: broadcast output in every pane" zellij-2 "zellij-fsn1-pve1" "zellij-fsn1-pve2" "zellij-fsn1-pve3"
|
||||
zellij kill-all-sessions --yes >/dev/null 2>&1
|
||||
|
||||
# --- TUI: mark hosts, open them together ---
|
||||
term tmux new-session -s tui netssh
|
||||
visible "TUI: search screen" tui-1 "Search hostname"
|
||||
tmux send-keys -t tui "pve"
|
||||
visible "TUI: results listed" tui-2 "$p1" "$p2" "$p3"
|
||||
tmux send-keys -t tui C-a
|
||||
visible "TUI: all marked" tui-3 "3 marked"
|
||||
tmux send-keys -t tui Enter
|
||||
visible "TUI: marked hosts open side by side" tui-4 "fsn1-pve1:~\$" "fsn1-pve2:~\$" "fsn1-pve3:~\$"
|
||||
tmux kill-server
|
||||
|
||||
stop_screen
|
||||
@@ -76,3 +76,79 @@ wait_for() {
|
||||
sleep 0.2
|
||||
done
|
||||
}
|
||||
|
||||
# --- rendering in a real terminal emulator (tests/16-render.sh) ---
|
||||
# Xvfb provides a screen, xterm renders the terminal, `import` takes a
|
||||
# screenshot, and tesseract reads it back, so checks see what a user sees.
|
||||
|
||||
start_screen() {
|
||||
Xvfb :99 -screen 0 1920x1080x24 -nolisten tcp >/dev/null 2>&1 &
|
||||
XVFB_PID=$!
|
||||
export DISPLAY=:99
|
||||
wait_for 10 import -window root -crop 1x1+0+0 null:
|
||||
}
|
||||
|
||||
stop_screen() {
|
||||
pkill -u "$(id -u)" xterm 2>/dev/null
|
||||
kill "$XVFB_PID" 2>/dev/null
|
||||
wait "$XVFB_PID" 2>/dev/null
|
||||
}
|
||||
|
||||
# term CMD... — run CMD in a full-screen xterm (UTF-8, DejaVu Sans Mono).
|
||||
term() {
|
||||
pkill -u "$(id -u)" xterm 2>/dev/null
|
||||
xterm -u8 -geometry 190x52+0+0 -fa 'DejaVu Sans Mono' -fs 11 -bg '#1e1e1e' -fg '#d4d4d4' \
|
||||
-T netssh -e "$@" >/dev/null 2>&1 &
|
||||
sleep 1
|
||||
}
|
||||
|
||||
# shot NAME — screenshot to /artifacts/NAME.png.
|
||||
shot() { import -window root "/artifacts/$1.png"; }
|
||||
|
||||
# OCR preprocessing variants: no single one reads everything (dim hint lines,
|
||||
# red/green borders): plain luminance, stretched dark range for dim gray, and
|
||||
# the brightest color channel for colored text.
|
||||
OCR_VARIANTS=("-colorspace gray" "-colorspace gray -level 10%,40%"
|
||||
"-channel RGB -separate -evaluate-sequence max -level 10%,60%")
|
||||
|
||||
# ocr_variant NAME I — OCR of screenshot NAME with preprocessing variant I.
|
||||
ocr_variant() {
|
||||
# shellcheck disable=SC2086
|
||||
magick "/artifacts/$1.png" ${OCR_VARIANTS[$2]} -negate -resize 300% png:- |
|
||||
tesseract stdin stdout --psm 6 2>/dev/null
|
||||
}
|
||||
|
||||
# screen_text NAME — the text of all variants combined.
|
||||
screen_text() {
|
||||
local i
|
||||
for i in "${!OCR_VARIANTS[@]}"; do ocr_variant "$1" "$i"; done
|
||||
}
|
||||
|
||||
# ocr_norm maps characters OCR confuses in DejaVu Sans Mono (1/l/I/|, C/( , 0/O)
|
||||
# to one form; expected text gets the same mapping, so comparisons stay strict
|
||||
# about everything else.
|
||||
ocr_norm() { tr 'lI|(O' '111C0'; }
|
||||
|
||||
# screen_shows NAME TEXT... — take a screenshot and check that OCR finds every TEXT.
|
||||
screen_shows() {
|
||||
local name=$1 text want
|
||||
shift
|
||||
shot "$name"
|
||||
text=$(screen_text "$name" | ocr_norm)
|
||||
for want in "$@"; do
|
||||
want=$(printf '%s' "$want" | ocr_norm)
|
||||
[[ $text == *"$want"* ]] || return 1
|
||||
done
|
||||
}
|
||||
|
||||
# screen_count NAME TEXT — how often TEXT is on screenshot NAME: the highest
|
||||
# count of any OCR variant (they read the same screen, so they are not added).
|
||||
screen_count() {
|
||||
local want i n max=0
|
||||
want=$(printf '%s' "$2" | ocr_norm)
|
||||
for i in "${!OCR_VARIANTS[@]}"; do
|
||||
n=$(ocr_variant "$1" "$i" | ocr_norm | grep -oF -- "$want" | wc -l)
|
||||
[ "$n" -gt "$max" ] && max=$n
|
||||
done
|
||||
echo "$max"
|
||||
}
|
||||
+2
-2
@@ -5,8 +5,8 @@ set -euo pipefail
|
||||
cd "$(dirname "$0")"
|
||||
compose=(docker compose -f compose.yaml)
|
||||
|
||||
mkdir -p .keys .state
|
||||
chmod a+rwx .state # written by the seed container's user
|
||||
mkdir -p .keys .state .artifacts
|
||||
chmod a+rwx .state .artifacts # written by the container user (screenshots in .artifacts)
|
||||
if [ ! -f .keys/id_ed25519 ]; then
|
||||
ssh-keygen -q -t ed25519 -N '' -C netssh-test -f .keys/id_ed25519
|
||||
cp .keys/id_ed25519.pub .keys/authorized_keys
|
||||
|
||||
Reference in new issue
Block a user