| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244 |
- <?php
- /**
- * One cron line for every background job, for hosts that do have cron.
- *
- * Why this exists next to the web-driven flow
- * ------------------------------------------
- * The application assumes no cron and drives itself: a public page render calls
- * archive_kick() and a chain of worker.php requests builds the gallery archives
- * (see app/archive.php); an admin page render calls manage_kick() and
- * manage-worker.php makes the scheduled backup and sends the heartbeat (see
- * app/manage.php). That works, but it moves one slice or one job per firing and
- * only while somebody is looking at the site.
- *
- * Where real cron exists it can do better, and cron.php is the single entry
- * point for it: one invocation drains *everything* outstanding — every settled
- * gallery, not just the first, plus whatever the schedule owes — bounded only by
- * cron.max_seconds.
- *
- * Running alongside the web flow, not instead of it
- * ------------------------------------------------
- * Nothing here switches the kicks off. Both drivers stay live and may run at the
- * same moment, because the locks they already share decide who does what:
- *
- * - data/archive.lock — exactly one process works on an archive. Taken and
- * released again *per slice* rather than held across the drain, so a cron
- * busy for four minutes never starves the worker chain and never makes an
- * admin's "Rebuild now" (which waits only archive_lock(15)) fail.
- * - data/manage.lock — one scheduled backup at a time, the same lock
- * manage-worker.php takes. Held elsewhere means somebody else is on it, so
- * this run leaves it alone.
- * - data/cron.lock — one cron invocation at a time. A gallery that takes
- * longer to archive than the cron interval must not accumulate a queue of
- * invocations waiting behind it.
- *
- * Off by default: cron.enabled has to be set. A copy of this site restored onto
- * a staging host brings the crontab with it, and it must not start rebuilding
- * archives and uploading backups on its own.
- *
- * What it deliberately does not do is update the software — same reason as
- * app/manage.php: that belongs in front of a human, at Admin -> Maintenance.
- */
- declare(strict_types=1);
- /** Whether cron.php is allowed to do anything. Opt-in; see the header. */
- function cron_enabled(): bool
- {
- return (bool)config('cron.enabled', false);
- }
- /**
- * Seconds of work per invocation. Floored at 30 because a budget below one
- * archive slice would make no progress at all on a large gallery.
- */
- function cron_max_seconds(): int
- {
- return max(30, (int)config('cron.max_seconds', 240));
- }
- /** One invocation at a time. */
- function cron_lock_file(): string
- {
- return DATA_DIR . '/cron.lock';
- }
- /** What the last run did, for Admin -> Maintenance -> Schedule. */
- function cron_state_file(): string
- {
- return DATA_DIR . '/cron.json';
- }
- /** Last run: ['last_run_at' => int, 'duration' => float, 'jobs' => [], 'remaining' => int]. */
- function cron_state(): array
- {
- return json_read(cron_state_file());
- }
- /**
- * Galleries still waiting in the archive queue, settled or not. Reported at the
- * end of a run so a cron that ran out of budget says so.
- */
- function cron_queued_count(): int
- {
- $queue = json_read(archive_queue_file());
- return count($queue['galleries'] ?? []);
- }
- /**
- * The next settled gallery that is not in $skip.
- *
- * archive_next_due() always returns the *first* settled entry, which is right
- * for a worker that only ever does one slice but would hand this loop the same
- * failing gallery forever. Same settle test, and deliberately a separate
- * function: archive_next_due() is part of worker.php's contract.
- *
- * @param array<string,bool> $skip
- */
- function cron_next_due(array $skip): ?string
- {
- $queue = json_read(archive_queue_file());
- $settle = (int)config('archive.settle_seconds', 300);
- foreach ($queue['galleries'] ?? [] as $slug => $dirtyAt) {
- if (isset($skip[(string)$slug])) {
- continue;
- }
- if (time() - (int)$dirtyAt >= $settle) {
- return (string)$slug;
- }
- }
- return null;
- }
- /**
- * Run the scheduled backup and heartbeat, if either is due and nobody else is
- * already on them. The body is manage-worker.php's, sharing its lock.
- *
- * @return string[] a line per job, for the log and the cron mail
- */
- function cron_run_manage(): array
- {
- if (!manage_available()) {
- return [];
- }
- $lock = fopen(DATA_DIR . '/manage.lock', 'c');
- if ($lock === false || !flock($lock, LOCK_EX | LOCK_NB)) {
- return []; // a manage_kick() from the backoffice has it; leave quietly
- }
- try {
- // Never throws: every job inside reports its own failure as a line.
- return manage_run_due();
- } finally {
- flock($lock, LOCK_UN);
- fclose($lock);
- }
- }
- /**
- * Build every gallery archive that is due, until the queue is drained or the
- * budget runs out.
- *
- * The budget is checked before each slice, never inside one: archive_run_slice()
- * stops *starting* photos at archive.step_seconds but still has to finish the one
- * in flight, so a single large original can overrun the total. A soft ceiling is
- * the only honest kind here, which is why cron.max_seconds defaults well below a
- * URL fetcher's timeout.
- *
- * @return string[] a line per finished or abandoned archive
- */
- function cron_run_archives(float $deadline): array
- {
- $lines = [];
- $skip = []; // slug => true, for this invocation only
- $progress = []; // slug => photos done when last seen, to catch a stall
- $step = (float)config('archive.step_seconds', 25);
- $worked = false;
- while (microtime(true) < $deadline) {
- $slug = cron_next_due($skip);
- if ($slug === null) {
- break; // nothing settled; the next tick will look again
- }
- // Non-blocking: a worker or an admin holding it is already doing this
- // work, and there is nothing useful to do meanwhile.
- $lock = archive_lock();
- if ($lock === null) {
- $lines[] = 'archive: another worker holds the lock';
- break;
- }
- try {
- $budget = min($step, $deadline - microtime(true));
- $result = archive_run_slice($slug, max(1.0, $budget));
- } finally {
- flock($lock, LOCK_UN);
- fclose($lock);
- }
- $worked = true;
- if ($result['error'] !== null) {
- // Left queued on purpose where the slice left it queued: the next
- // invocation retries, this one stops paying for it.
- $skip[$slug] = true;
- $lines[] = 'archive ' . $slug . ' failed: ' . $result['error'];
- continue;
- }
- if ($result['finished']) {
- $lines[] = 'archive ' . $slug . ' built, ' . $result['total'] . ' photos'
- . ($result['size'] !== null ? ', ' . human_bytes((int)$result['size']) : '');
- // Not unqueued when the gallery changed under the build (see
- // archive_finish): it is due again immediately and the loop rebuilds
- // it, which is the point of draining.
- continue;
- }
- if ($result['done'] <= ($progress[$slug] ?? -1)) {
- // A slice that copied nothing and reported no error would spin.
- $skip[$slug] = true;
- $lines[] = 'archive ' . $slug . ' made no progress; skipped';
- continue;
- }
- $progress[$slug] = (int)$result['done'];
- }
- if ($worked) {
- // The queue is empty, or as empty as this run can make it. Reset the
- // chain counter the same way worker.php does when a chain ends, so one
- // left high by an interrupted chain cannot trip archive.max_chain on the
- // next web-driven build.
- if (cron_queued_count() === 0) {
- json_update(archive_queue_file(), function (array $queue): array {
- $queue['chain'] = 0;
- return $queue;
- });
- }
- // The next page view can skip its dispatch: whatever it would have
- // started has just been done.
- @touch(archive_tick_file());
- }
- return $lines;
- }
- /**
- * Everything, in order: the cheap bounded schedule first, then the open-ended
- * archive drain with whatever budget is left.
- *
- * @return array{jobs: string[], remaining: int, duration: float}
- */
- function cron_run_all(?float $budget = null): array
- {
- $started = microtime(true);
- $budget ??= (float)cron_max_seconds();
- $deadline = $started + $budget;
- $jobs = cron_run_manage();
- foreach (cron_run_archives($deadline) as $line) {
- $jobs[] = $line;
- }
- return [
- 'jobs' => $jobs,
- 'remaining' => cron_queued_count(),
- 'duration' => microtime(true) - $started,
- ];
- }
|