fix(logging): demote per-iteration cron logs from info to debug #1

Merged

aureus merged 24 commits from log-levels-audit into main 2026-05-25 02:58:41 +00:00

Conversation 0 Commits 24 Files changed 30 +271 -54

aureus commented 2026-05-24 07:16:58 +00:00

Owner

Copy link

Top-level cron start/done markers stay at info. Per-user/per-audience
inner events (syncHistory start/done, refreshForUser start/skip/done)
and internal pruneCache row-count stats drop to debug so production
logs at the default level are not flooded during batch operations.

Top-level cron start/done markers stay at info. Per-user/per-audience inner events (syncHistory start/done, refreshForUser start/skip/done) and internal pruneCache row-count stats drop to debug so production logs at the default level are not flooded during batch operations.

aureus added 1 commit 2026-05-24 07:16:59 +00:00

fix(logging): demote per-iteration cron logs from info to debug ... be1905fa89

Top-level cron start/done markers stay at info. Per-user/per-audience
inner events (syncHistory start/done, refreshForUser start/skip/done)
and internal pruneCache row-count stats drop to debug so production
logs at the default level are not flooded during batch operations.

aureus added 1 commit 2026-05-24 07:22:23 +00:00

fix(ci): add python3/make/g++ to CI image for native addon compilation ... d603eb3f34

better-sqlite3 uses node-gyp as a fallback when prebuild-install
times out. node-gyp requires Python and build tools which are absent
from node:24-bookworm-slim.

aureus added 1 commit 2026-05-24 07:35:45 +00:00

fix(ui): add count badge to curate_body Movies panel ... 86313e98c8

movieCount was computed and passed to the template but never rendered.
Add a rounded-full badge showing the queue depth at the top of the
Movies panel when movieCount > 0, matching the existing test assertions
in feed-tabs.test.ts.

aureus added 1 commit 2026-05-24 07:41:52 +00:00

fix(ci): use Actions services for PG/MariaDB, fix container networking ... bf39bf042c

When the job runs inside a container (container: image:), docker compose
spawned databases are on the host — localhost inside the container doesn't
reach them. Switch to Actions services: which run as sidecars on the same
network as the job container, reachable by service name.

Add TEST_MYSQL_HOST/TEST_MYSQL_PORT env var overrides to helpers.ts
(postgres already had TEST_DB_URL). Set both in the test steps so the
correct service hostnames are used in CI without changing dev defaults.

aureus added 1 commit 2026-05-24 14:56:56 +00:00

fix(tests): make PG and MariaDB integration tests reliable ... ffff899e8c

Five root causes, all fixed:

1. Parallel file execution on shared databases — vitest.config.ts now
   sets fileParallelism=false when TEST_DB_CLIENT is pg or mysql2,
   serialising test files to prevent concurrent migration races and
   duplicate-key seed collisions.

2. State pollution between files — setupTestDb now truncates all data
   tables before re-seeding for both PG (TRUNCATE … RESTART IDENTITY
   CASCADE) and MySQL (TRUNCATE with FK checks disabled).  For PG,
   setval() advances serial sequences past the explicitly-seeded ids so
   the next auto-generated insert doesn't collide at 1.  For MySQL, the
   first call per file drops schema + knex metadata tables then
   re-applies migrations, guarding against corrupt state left by
   previous parallel runs.

3. const [id] = db.insert() broken on PG (Knex 3.x returns the raw pg
   result object, which is non-iterable) — fixed in auth.ts
   (upsertUser, createUser, initUserAudiences), admin.ts, and
   settings.ts by inserting then immediately re-selecting by unique
   field to obtain the auto-generated id.  Same pattern fixed in
   settings.test.ts test setup.

4. Migration FK type mismatch on MariaDB 11 — t.increments() produces
   INT UNSIGNED but FK reference columns used t.integer() (signed INT).
   MariaDB 11 rejects signed→unsigned FK constraints with errno 150.
   Added .unsigned() to all 8 FK columns in 001_initial.ts and the
   restored column in 002_simplify_ratings.ts down().  PG and SQLite
   silently ignore .unsigned().

5. ISO 8601 timestamps rejected by MySQL — test inserts for
   media_metadata.fetched_at and library.synced_at passed
   new Date().toISOString() which MySQL TIMESTAMP columns refuse.
   Dropped the explicit field from those inserts so the column default
   applies.

aureus force-pushed log-levels-audit from ffff899e8c to 95ade8393b 2026-05-24 14:57:13 +00:00 Compare

aureus added 1 commit 2026-05-24 15:08:45 +00:00

chore: pin jellyfin in docker-compose.dev.yml c25e2ad2e8

aureus added 1 commit 2026-05-24 15:21:31 +00:00

try security_opt 74a85161da

aureus added 1 commit 2026-05-24 15:52:33 +00:00

fix(ci): add apparmor:unconfined to jellyfin in dev compose ... 42d5a5429e

seccomp:unconfined alone didn't prevent the exit 139 (SIGSEGV) in CI.
AppArmor runs independently of seccomp on Ubuntu hosts; the docker-default
profile can still block syscalls even when seccomp is disabled. Adding
apparmor:unconfined removes that restriction for the Jellyfin container.

aureus added 1 commit 2026-05-24 16:27:51 +00:00

fix(rate): explicit merge columns to fix MariaDB upsert ... 96e8fb348b

Knex's .merge() with no args includes key columns (user_id, jellyfin_id)
in the ON DUPLICATE KEY UPDATE clause. On MariaDB 11 this causes a 1062
duplicate-key error when a conflict is triggered, because MariaDB checks
uniqueness even when the key columns are being set to their existing values.
The route's catch block then returns 500, leaving the old rating in place.

Fix: pass explicit non-key columns to .merge() so the UPDATE clause only
touches the data columns (rating, tmdb_id, media_type / play_count,
is_favorite). Same pattern applied to both ratings upsert sites and
the watch_history upsert.

aureus added 2 commits 2026-05-24 17:14:34 +00:00

fix(rate): explicit merge columns to fix MariaDB upsert ... 17a2ad0a24

Knex's .merge() with no args includes key columns (user_id, jellyfin_id)
in the ON DUPLICATE KEY UPDATE clause. On MariaDB 11 this causes a 1062
duplicate-key error when a conflict is triggered, because MariaDB checks
uniqueness even when the key columns are being set to their existing values.
The route's catch block then returns 500, leaving the old rating in place.

Fix: pass explicit non-key columns to .merge() so the UPDATE clause only
touches the data columns (rating, tmdb_id, media_type / play_count,
is_favorite). Same pattern applied to both ratings upsert sites and
the watch_history upsert.

chore: pin container images to LTS/specific versions ... dd386c6282

mariadb:11 is a rolling tag that resolves to different minor versions
across CI runners and developer machines. mariadb:11.4 is the current
LTS branch; pinning it ensures CI and dev always run the same engine.

Also pin seerr to v3.2.0 (was :latest) in dev compose.

aureus added 1 commit 2026-05-24 17:26:56 +00:00

fix(ci): add CI-only compose overlay for Jellyfin .NET JIT flags ... 8b6dbdeb33

Jellyfin 10.x / .NET 8 JIT emits AVX/AVX2/SSSE3 instructions on startup.
The CI runners live inside Proxmox VMs (nested virt) where those instruction
sets are not reliably exposed, causing an immediate SIGSEGV (exit 139).

Rather than pollute the dev compose with CI-specific workarounds, add a
separate dev/docker-compose.ci.yml override that is merged in only by the
pr-checks workflow via COMPOSE_FILE. Local dev is unaffected.

aureus added 1 commit 2026-05-24 19:15:10 +00:00

fix(tests): eliminate MariaDB 11.4 race in rate integration tests ... ebc7b93361

The two 're-rating' tests were failing on MariaDB 11.4 with either a
wrong rating value (0 instead of 2) or zero rows returned.  Root cause:
triggerRefreshAllAudiences fires fire-and-forget after res.send(), and
on MariaDB 11.4 its concurrent SELECT on the ratings table conflicts with
the second request's INSERT ON DUPLICATE KEY UPDATE or check-then-update.

Two complementary fixes:

1. pool: { min: 0, max: 1 } in the mysql2 test Knex config.
   Serialises all DB access so the background refresh's queries cannot
   run concurrently with the second request's writes.  The same-SQL
   template with different bindings (rating=0 vs rating=2) executes on
   a single connection, eliminating any per-connection prepared-statement
   interleaving that triggered MariaDB 11.4's misbehaviour.

2. drainPool() helper + usage in afterEach and between requests.
   Waits for the pool to go idle (numUsed === 0) before the pool is
   destroyed.  This ensures background reads have released their
   connections (and their MariaDB metadata locks) before the next
   test's TRUNCATE TABLE statements run.

aureus added 1 commit 2026-05-24 19:23:37 +00:00

try a jellyfin fix yet again 1444a09db1

aureus added 1 commit 2026-05-24 19:41:23 +00:00

fix(tests): drain pool before destroy in all integration afterEach hooks ... 891001fd30

settings.test.ts was missing drainPool() before db.destroy(), leaving a
live MariaDB metadata lock when POST /audiences/:id/members fired its
background refreshForUser() call. The next test's beforeEach TRUNCATE
blocked on that lock; under CI load the 10 s hookTimeout was exceeded.

Added drainPool() to all five integration test files that lacked it
(settings, admin, auth-routes, discover, rec-index). rate.test.ts already
had the fix; this brings all files into alignment.

Also added a convention comment in helpers.ts at makeTestDb() documenting
the invariant, raised hookTimeout to 30 s for mysql2 in vitest.config.ts
as a safety net, and added DOTNET_USE_POLLING_FILE_WATCHER=1 to the CI
overlay to address the Jellyfin inotify-limit crash (exit 139) — the
actual error exposed by the new diagnostic step; the existing JIT flags
remain as defense-in-depth.

aureus added 1 commit 2026-05-24 19:58:27 +00:00

fix(bootstrap): switch wizard readiness check to /health, increase all timeouts ... fce9d8d763

/Startup/Configuration returns non-200 on Jellyfin 10.10.7 fresh installs in
the CI environment (Proxmox nested-virt), causing a 2-minute timeout even though
Kestrel is already accepting connections.  /health is reliable from the moment
Kestrel binds and is the correct readiness signal.

All bootstrap timeouts doubled/more to accommodate the slow CI hardware:
- waitFor default: 120s → 300s
- library scan:    300s → 600s
- TMDB metadata:  180s → 600s
- CI snapshot step: 25min → 40min

aureus added 2 commits 2026-05-24 20:24:09 +00:00

fix(bootstrap): accept 404 from /Startup/Configuration as a ready state ... 634064b146

In Jellyfin 10.10.x, a fresh install may pre-initialise the wizard state and
return 404 from /Startup/Configuration instead of 200.  The previous code only
accepted r.ok (200-299), causing a 2-minute timeout even though Jellyfin was
fully up and ready.

404 is now treated as "ready": the wizard state is determinable, and the
StartupWizardCompleted check that follows handles both cases correctly.

fix(ci): connect CI container to dev network so bootstrap can reach services ... 9f08d3ec2e

The CI job runs inside the watchcraft-ci container (selfhosted-config_runner
network).  dev:up creates Jellyfin/Seerr on the dev_default network — a
completely separate Docker network.  Every fetch('http://localhost:18096/...')
inside bootstrap.ts got ECONNREFUSED, silently swallowed by .catch(()=>false),
causing the waitFor loop to spin for its entire timeout on every run.

Fix: split dev:reset into dev:reset:services (Docker up only) + bootstrap so
the CI workflow can do `docker network connect dev_default $(cat /etc/hostname)`
in between.  With the CI container on dev_default, bootstrap reaches services
by container name.  JELLYFIN_BOOTSTRAP_HOST and SEERR_BOOTSTRAP_HOST env vars
override the localhost defaults in bootstrap.ts for this path.

aureus added 1 commit 2026-05-24 20:45:59 +00:00

fix(ci): reach dev services via Docker host gateway instead of network connect ... 2c9f70667c

The previous approach of joining the CI container to dev_default via
`docker network connect $(cat /etc/hostname)` fails because the CI
container's hostname is not its Docker container ID as seen by the daemon.

Instead, compute the Docker host gateway IP at runtime with ip-route and
reach the already host-mapped ports (18096, 15055) directly. This avoids
needing to know the CI container's identity at all and removes the fragile
network-connect step.

aureus added 1 commit 2026-05-24 20:52:30 +00:00

fix(ci): read gateway from /proc/net/route — ip command not available in CI image ... 5a0af2fe6c

iproute2 is not installed in the watchcraft-ci image. Parse /proc/net/route
directly with awk instead: the default-route row (Destination=00000000) holds
the gateway as a little-endian hex field, no external tools needed.

aureus added 1 commit 2026-05-24 21:04:27 +00:00

fix(ci): drop strtonum — use bash hex arithmetic to decode gateway ... 77c1566b77

strtonum is gawk-only; the CI image has mawk/busybox awk. Let awk extract
the raw hex field and convert each byte with bash's $((16#XX)) arithmetic
instead, which works in any POSIX shell without gawk extensions.

aureus added 1 commit 2026-05-25 00:24:51 +00:00

fix(ci): connect dev containers to runner network; expand failure diagnostics ... 9382b11d07

Network fix: instead of connecting the CI container to dev_default (which
requires knowing its Docker ID — unreliable in this runner), connect the dev
containers (watchcraft-jellyfin, watchcraft-seerr) to selfhosted-config_runner
after they start. The CI container already lives on that network, so Docker DNS
resolves container names immediately after the connect. No /proc parsing or
host-gateway hacks needed.

Diagnostics: expand the on-failure dump to capture /etc/hosts, the raw routing
table, both Docker networks (selfhosted-config_runner and dev_default) with full
membership lists, live curl probes to watchcraft-jellyfin, Seerr logs, and the
most recent Jellyfin log file from dev/runtime/jellyfin/log/.

aureus added 1 commit 2026-05-25 01:34:25 +00:00

fix(ci): reach dev containers via host-mapped ports instead of network join ... 58f80ef470

selfhosted-config_runner lives on the outer Forgejo runner daemon; the HOST
Docker daemon (used by docker compose) has no knowledge of it. docker network
connect selfhosted-config_runner always fails with "network not found".

Instead, compute the HOST IP from /proc/net/route (default gateway of the CI
container's eth0), then reach Jellyfin/Seerr via their host-mapped ports
(18096, 15055). Docker's iptables DNAT rules on the HOST forward the traffic
to the correct containers on dev_default.

Also update the diagnostics step to test connectivity via HOST_IP:port rather
than the unresolvable container hostnames.

aureus force-pushed log-levels-audit from 58f80ef470 to 8ade746af9 2026-05-25 01:39:58 +00:00 Compare

aureus added 1 commit 2026-05-25 02:57:38 +00:00

fix(ci): run bootstrap inside dev_default; fix Seerr perms; update snapshot baselines ... 18633c2e3f

Two CI bootstrap failures:

1. Network: the CI container and HOST Docker daemon are on separate daemon
   contexts. The gateway IP approach (172.25.0.1:18096) gives Connection
   refused — Docker's DNAT rules don't fire for traffic on that bridge path.
   Fix: run the bootstrap process inside a transient container on dev_default
   via docker run --network, where it reaches services by DNS name.

2. Seerr EACCES: Seerr starts as a non-root user and can't mkdir
   /app/config/logs/ when the bind-mounted runtime dir is root-owned.
   Fix: add chmod -R a+rwx to dev:up so dirs are world-writable before
   containers start.

Also update the diagnostics connectivity probe to run from a transient
alpine container on dev_default rather than probing a host-gateway IP.

Browser test fixes:
- snapshots.test.ts: reset audience selector to Solo before taking the
  discover-mode screenshot, preventing session leakage from the
  audience-switching tests in discover.test.ts.
- Updated baselines for tiers where the curate badge (added earlier on
  this branch) or audience state caused snapshot mismatches.

aureus merged commit 920981ca69 into main 2026-05-25 02:58:41 +00:00

aureus referenced this pull request from a commit 2026-05-25 02:58:43 +00:00

fix(logging): demote per-iteration cron logs from info to debug (#1)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

aureus/watchcraft!1

No description provided.