SupplyCore Console

Log Viewer

System health at a glance

Monitor job runs, failures, timeouts, and external service connectivity. Fix issues before they cascade.

Live refresh · 2h ago

Market anomalies

3 active alerts

View all

Critical misprice Alliance Market

Plutonium Charge S

Current 0.02 ISK · Normal 833.01 ISK · 0.00% of normal

1035466617946 · Fresh 2m ago

Critical misprice Reference Hub

Nanofiber Internal Structure I

Current 4,656.00 ISK · Normal 186,681,434.85 ISK · 0.00% of normal

60003760 · Fresh 1d ago

Critical misprice Reference Hub

Microwave M

Current 5,900.00 ISK · Normal 15,263,812.55 ISK · 0.00% of normal

60003760 · Fresh 2m ago

Needs attention

All systems operational.

Healthy jobs

Jobs that completed their last run successfully.

Never ran

Enabled jobs that have never executed.

Enabled / Total

65 / 67

Active scheduled jobs out of total registered.

Backlog Overview

Queued

Jobs waiting to be picked up by a worker.

Running

Jobs currently executing across all workers.

Retry

Jobs waiting to retry after a transient failure.

External Services

Online Neo4j

1ms · 2026.02.3

Online InfluxDB

0ms · v2.8.0

Stuck / Timed-out Runs

Job	Stuck instances	Oldest	Newest	Running for
killmail.r2z2.stream	29	2026-03-31 21:08:27 UTC	2026-04-03 13:06:40 UTC	2d

Recent Failures (last 24 h)

Job	Occurrences	Last seen	First seen	Error
scheduler.job.theater_suspicion	9×	8m ago	19h ago	OperationalError: (2003, "Can't connect to MySQL server on '127.0.0.1' ([Errno 104] Connection reset by peer)")
scheduler.job.evewho_alliance_member_sync	1×	28m ago	28m ago	(2006, "MySQL server has gone away (ConnectionResetError(104, 'Connection reset by peer'))")
scheduler.job.evewho_alliance_member_sync	2×	46m ago	3h ago	(2003, "Can't connect to MySQL server on '127.0.0.1' ([Errno 104] Connection reset by peer)")
scheduler.job.compute_alliance_dossiers	2×	59m ago	19h ago	OperationalError: (2003, "Can't connect to MySQL server on '127.0.0.1' ([Errno 104] Connection reset by peer)")
scheduler.job.theater_clustering	1×	2h ago	2h ago	RemoteDisconnected: Remote end closed connection without response
scheduler.job.theater_graph_integration	8×	2h ago	21h ago	OperationalError: (2006, "MySQL server has gone away (ConnectionResetError(104, 'Connection reset by peer'))")
scheduler.job.graph_motif_detection_sync	186×	4h ago	10h ago	Detectors failed: triangle
scheduler.job.compute_graph_sync	33×	5h ago	21h ago	OperationalError: (1213, 'Deadlock found when trying to get lock; try restarting transaction')
scheduler.job.compute_alliance_dossiers	2×	5h ago	15h ago	OperationalError: (2006, "MySQL server has gone away (ConnectionResetError(104, 'Connection reset by peer'))")
scheduler.job.theater_graph_integration	6×	5h ago	20h ago	OperationalError: (2003, "Can't connect to MySQL server on '127.0.0.1' ([Errno 104] Connection reset by peer)")
scheduler.job.compute_graph_topology_metrics	20×	5h ago	20h ago	OperationalError: (1213, 'Deadlock found when trying to get lock; try restarting transaction')
scheduler.job.theater_suspicion	4×	6h ago	16h ago	OperationalError: (2006, "MySQL server has gone away (ConnectionResetError(104, 'Connection reset by peer'))")
scheduler.job.compute_economic_warfare	12×	7h ago	21h ago	OperationalError: (2013, 'Lost connection to MySQL server during query ([Errno 104] Connection reset by peer)')

All Jobs

All (67) Failed (0) Timeout (0) Overdue (0) Never ran (0) Healthy (65) Disabled

Job	Status	Last run	Last success	Interval	Pressure	Issue
Activity Priority Summary activity_priority_summary_sync	Healthy	1m ago	1m ago	15m	Healthy	Refreshed doctrine activity snapshots with 114 upserts.
Alliance Current alliance_current_sync	Healthy	1m ago	1m ago	4m	Healthy	Ingested/projected alliance structure orders (0 writes across 1 structures).
Alliance Historical alliance_historical_sync	Healthy	1m ago	1m ago	6h	Healthy	Materialized 0 alliance_structure historical snapshot rows.
Analytics Buckets (1d) analytics_bucket_1d_sync	Healthy	1m ago	1m ago	1h	Healthy	Upserted 0 stock and 0 price daily bucket rows.
Analytics Buckets (1h) analytics_bucket_1h_sync	Healthy	1m ago	1m ago	15m	Healthy	Upserted 15963 stock and 15963 price hourly bucket rows.
Battle Type Classification battle_type_classification	Healthy	1m ago	1m ago	15m	Healthy	battle_type_classification completed with status success.
Cache Expiry Cleanup cache_expiry_cleanup_sync	Healthy	1m ago	1m ago	1h	Healthy	Deleted 0 expired cache rows (adj snapshots: 0).
Alliance Dossiers compute_alliance_dossiers	Healthy	2m ago	2m ago	30m	Healthy	Computed 386 alliance dossiers.
Alliance Relationships compute_alliance_relationships	Healthy	1m ago	1m ago	30m	Healthy	No new killmails to process for alliance relationships.
Battle Actor Features compute_battle_actor_features	Healthy	2m ago	2m ago	10m	Healthy	Extracted actor features for 19471 participants, wrote 19471 rows.
Battle Anomalies compute_battle_anomalies	Healthy	2m ago	2m ago	10m	Healthy	Detected 3509 anomalies (84 high-sustain) from 3509 rows.
Battle Rollups compute_battle_rollups	Healthy	36s ago	36s ago	10m	Healthy	Rolled up 13604 killmails into battles across 14 batches.
Battle Target Metrics compute_battle_target_metrics	Healthy	2m ago	2m ago	10m	Healthy	Scored 633 target metrics across 1 batches (4 unscored).
Compute Buy All compute_buy_all	Healthy	2m ago	2m ago	15m	Healthy	Precomputed 4 buy-all request(s) with 480 item rows.
Character Feature Windows compute_character_feature_windows	Healthy	2m ago	2m ago	30m	Healthy	Computed feature windows across 0 batches, wrote 0 feature rows.
Cohort Baselines compute_cohort_baselines	Healthy	2m ago	2m ago	30m	Healthy	Assigned 15975 cohort memberships across 5 cohorts for 8959 characters. Computed 45 feature baselines. Updated 8959 cohort-relative scores.
Co-Presence Edges compute_copresence_edges	Healthy	2m ago	2m ago	1h	Healthy	No characters to process.
Counterintel Pipeline compute_counterintel_pipeline	Healthy	1m ago	1m ago	15m	Healthy	Processed 0 eligible 100+ participant battles across 0 batches.
Economic Warfare compute_economic_warfare	Healthy	2m ago	2m ago	30m	Healthy	Economic warfare: 1273 fit families, 1421 modules scored.
Graph Derived Relationships compute_graph_derived_relationships	Healthy	1m ago	1m ago	15m	Healthy	compute_graph_derived_relationships completed successfully.
Graph Insights compute_graph_insights	Healthy	2m ago	2m ago	15m	Healthy	compute_graph_insights completed successfully.
Graph Sync compute_graph_sync	Healthy	1m ago	1m ago	15m	Healthy	Graph sync pipeline (sync+derived+prune+topology) finished with status success.
Graph Killmail Edges compute_graph_sync_killmail_edges	Healthy	2m ago	2m ago	10m	Healthy	Synced 4 killmail edges (ATTACKED_ON + VICTIM_OF) into Neo4j.
Graph Killmail Entities compute_graph_sync_killmail_entities	Healthy	2m ago	2m ago	10m	Healthy	Projected 284 killmails into Neo4j.
Compute Signals compute_signals	Healthy	2m ago	2m ago	15m	Healthy	Generated 120 intelligence signals from 120 buy-all items.
Suspicion Scores compute_suspicion_scores	Healthy	1m ago	1m ago	10m	Healthy	Scored 9083 characters (8197 filtered below sample threshold).
Suspicion Scores V2 compute_suspicion_scores_v2	Healthy	1m ago	1m ago	15m	Healthy	Scored 8959 characters from 8959 behavioral baselines.
Threat Corridors compute_threat_corridors	Healthy	2m ago	2m ago	1h	Healthy	Found 200 threat corridors across 2803 active systems.
Corp Standings corp_standings_sync	Healthy	1m ago	1m ago	1h	Healthy	Skipped — no ESI OAuth token available.
Current-State Refresh current_state_refresh_sync	Healthy	1m ago	1m ago	12m	Healthy	Refreshed scheduler current-state rows for 134 jobs.
Dashboard Summary dashboard_summary_sync	Healthy	1m ago	1m ago	15m	Healthy	Refreshed dashboard summary intelligence snapshot.
Deal Alerts deal_alerts_sync	Healthy	2m ago	2m ago	5m	Healthy	Materialized 7204 active deal alerts.
Doctrine Intelligence doctrine_intelligence_sync	Healthy	1m ago	1m ago	15m	Healthy	Updated doctrine intelligence stock/activity rows (3964 upserts).
Entity Metadata Resolve entity_metadata_resolve_sync	Healthy	1m ago	1m ago	5m	Healthy	Resolved 500/500 pending entities (0 failed, 0 still pending).
Escalation Detection escalation_detection	Healthy	1m ago	1m ago	30m	Healthy	escalation_detection completed with status success.
ESI Alliance History esi_alliance_history_sync	Healthy	1m ago	1m ago	1m	Healthy	Fetched 5 characters, wrote 0 alliance history periods (0 errors, 0 skipped).
ESI Character Queue esi_character_queue_sync	Healthy	1m ago	1m ago	1m	Healthy	Queued 5 new attacker character IDs for ESI lookup (5 pending, 4322 done).
EveWho Alliance Member Sync evewho_alliance_member_sync	Healthy	3m ago	3m ago	30m	Healthy	Graph crawl: Phase 1 \| 1 alliances, 33 corps swept \| 58811 members graphed, 2863 queued \| 0 chars enriched, 0c/0a discovered \| 100/100 API calls, 589026ms
EveWho Enrichment Sync evewho_enrichment_sync	Healthy	2m ago	2m ago	10m	Healthy	Enriched 50/50 characters in 5 batch(es), 25036ms
Forecasting AI forecasting_ai_sync	Disabled	1m ago	1m ago	1h	Healthy	Refreshed forecasting snapshot with 300 candidates.
Graph Analyst Recalibration graph_analyst_recalibration	Healthy	1m ago	1m ago	2h	Healthy	Insufficient labels for recalibration (0/10).
Graph Community Detection graph_community_detection_sync	Healthy	2m ago	2m ago	1h	Healthy	Community detection: 204550 characters assigned to 201668 communities, 2110 bridges.
Graph Data Quality graph_data_quality_check	Healthy	2m ago	2m ago	1h	Healthy	Quality gate passed: score 0.6996 (204550 characters, 279 orphans, 1034815 dups).
Graph Evidence Paths graph_evidence_paths_sync	Healthy	2m ago	2m ago	1h	Healthy	No flagged characters above threshold.
Graph Model Audit graph_model_audit	Healthy	1m ago	1m ago	6h	Healthy	graph_model_audit completed with status success.
Graph Motif Detection graph_motif_detection_sync	Healthy	2m ago	2m ago	1h	Healthy	Motif detection: 802 motifs found (triangle=500, star=2, chain=300, fleet_core=0, rotating_scout=0).
Graph Temporal Metrics graph_temporal_metrics_sync	Healthy	1m ago	1m ago	30m	Healthy	Temporal metrics computed for 3 windows, 72 rows written, 0 drift flags.
Graph Typed Interactions graph_typed_interactions_sync	Healthy	1m ago	1m ago	30m	Healthy	Typed interactions: 0 Neo4j rels created, 0 MariaDB rows written.
Graph Universe graph_universe_sync	Healthy	1m ago	1m ago	6h	Healthy	Synced 114 regions, 1184 constellations, 8490 systems, 13968 stargates.
Intelligence Pipeline intelligence_pipeline	Healthy	1m ago	1m ago	30m	Healthy	Intelligence pipeline complete: 0 suspicion signals, 0 overlap records exported.
Jump Bridge Sync jump_bridge_sync	Healthy	2m ago	2m ago	1h	Healthy	jump_bridge_sync completed with status success.
Killmail zKB Repair killmail_zkb_repair	Healthy	1m ago	1m ago	1h	Healthy	killmail_zkb_repair completed with status success.
Loss Demand Summary loss_demand_summary_sync	Healthy	1m ago	1m ago	15m	Healthy	Refreshed loss-demand intelligence snapshot with 0 rows.
Market Comparison Summary market_comparison_summary_sync	Healthy	2m ago	2m ago	15m	Healthy	Market comparison: 4113 items evaluated (2617 in both markets).
Market Hub Current market_hub_current_sync	Healthy	1m ago	1m ago	8m	Healthy	Ingested/projected market-hub orders (32739 writes across 1 sources).
Market Hub Historical market_hub_historical_sync	Healthy	1m ago	1m ago	6h	Healthy	Materialized 0 market_hub historical snapshot rows.
Market Hub Local History market_hub_local_history_sync	Healthy	8m ago	8m ago	20m	Healthy	Market hub local history: 73262 daily candles from 3330723 snapshot metrics (30d window).
Pre-Op Join Detection pre_op_join_detection	Healthy	2m ago	2m ago	30m	Healthy	pre_op_join_detection completed with status success.
Rebuild AI Briefings rebuild_ai_briefings	Disabled	1m ago	1m ago	20m	Healthy	Rebuilt 228 doctrine AI briefings from deterministic fit activity inputs.
Shell Corp Detection shell_corp_detection	Healthy	1m ago	1m ago	1h	Healthy	shell_corp_detection completed with status success.
Staging System Detection staging_system_detection	Healthy	2m ago	2m ago	1h	Healthy	staging_system_detection completed with status success.
Temporal Behavior Detection temporal_behavior_detection	Healthy	2m ago	2m ago	30m	Healthy	Temporal behavior detection complete: 23524 characters analysed, 3852 evidence rows written, 0 Neo4j tags.
Theater Analysis theater_analysis	Healthy	2m ago	2m ago	30m	Healthy	Analyzed 609 theaters.
Theater Clustering theater_clustering	Healthy	8m ago	8m ago	30m	Healthy	Clustered 1290 battles into 609 theaters.
Theater Graph Integration theater_graph_integration	Healthy	2m ago	2m ago	1h	Healthy	Enriched 600 theaters with graph data.
Theater Suspicion theater_suspicion	Healthy	2m ago	2m ago	30m	Healthy	Integrated suspicion for 600 theaters.
Tracked Alliance Member Sync tracked_alliance_member_sync	Healthy	2m ago	2m ago	5m	Healthy	Tracked alliance sync \| 0 alliances, 0 corps \| 89497 members graphed, 50 ESI-enriched \| 100/300 ESI, 0/30 EveWho, 33569ms

Log Files

compute.log 247.8 MB

modified 2d ago

         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/pathlib.py", line 1015, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/var/www/SupplyCore/storage/run/worker-pool-heartbeat.json'

battle-intelligence.log 239.1 MB

modified 2s ago

{"event":"theater_suspicion.theater_done","timestamp":"2026-04-03T13:09:36.398181+00:00","theater_id":"019ede4d0157af9fa2afb8508579b6119824861634941bddc2fcc6d139101b3d","participants":10,"suspicious":2,"max_score":1.0}
{"event":"theater_suspicion.theater_done","timestamp":"2026-04-03T13:09:37.167567+00:00","theater_id":"364705a78f89a4a5e4a2d9411bb201243cb5c6c28ba17838edc872e5207feb86","participants":12,"suspicious":1,"max_score":0.6071}
{"event":"theater_suspicion.theater_done","timestamp":"2026-04-03T13:09:38.515418+00:00","theater_id":"03686c9a816e6ef96c93a231f4411c7b2aa96d7d945a0abe771a857a2a00c837","participants":31,"suspicious":7,"max_score":1.0}
{"event":"theater_suspicion.theater_done","timestamp":"2026-04-03T13:09:39.482275+00:00","theater_id":"ee35bd304f5f11fc748dc0e5ab20d16a083bb7f7dac0a6dd7d36dd016e619f0d","participants":10,"suspicious":3,"max_score":1.0}
{"event":"theater_suspicion.theater_done","timestamp":"2026-04-03T13:09:40.425103+00:00","theater_id":"25b011b07eb40159957101db504c8d902fab723efdd609ec0bf67fdee512d0ee","participants":17,"suspicious":0,"max_score":0.3962}

worker.log 168.4 MB

modified 13s ago

2026-04-03 15:08:57,641 INFO supplycore.economic_warfare Phase 3: scored 1421 unique modules
{"job_name":"compute_battle_rollups","duration_ms":63711,"rows_processed":13604,"rows_written":25971,"errors":"","computed_at":"2026-04-03 13:08:02","batch_count":14,"cursor_start":96740524,"cursor_end":132317794}
{"ts":"2026-04-03T13:09:06.628906+00:00","level":"info","logger":"supplycore.orchestrator","message":"job finished: compute_battle_rollups","event":"loop_runner.job_finished","job_key":"compute_battle_rollups","status":"success","duration_seconds":63.8,"error":null}
2026-04-03 15:09:17,769 INFO orchestrator.jobs.evewho_alliance_member_sync Corp 700801715: 4 members, 0 joined, 0 departed
2026-04-03 15:09:29,584 INFO orchestrator.jobs.evewho_alliance_member_sync Corp 707925643: 11 members, 0 joined, 0 departed

graph-sync.log 80.2 MB

modified 1m ago

{"event":"graph.derived.completed","timestamp":"2026-04-03T13:08:09.429469+00:00","status":"success","summary":"compute_graph_derived_relationships completed successfully.","started_at":"2026-04-03T13:08:09Z","finished_at":"2026-04-03T13:08:09Z","duration_ms":4951,"rows_seen":868,"rows_processed":868,"rows_written":868,"rows_skipped":0,"rows_failed":0,"batches_completed":0,"checkpoint_before":null,"checkpoint_after":null,"has_more":false,"error_text":null,"error":null,"warnings":[],"meta":{"job_key":"compute_graph_derived_relationships","schema_version":"job_result.v2","nodes_created":0,"nodes_merged":0,"relationships_created":868,"relationships_merged":0,"timestamp":"2026-04-03T13:08:09.429369+00:00","character_batches":10,"fit_batches":2,"character_batch_size":50,"fit_batch_size":200,"character_checkpoint":"2114764167","fit_checkpoint":"0"}}
{"event":"graph.prune.completed","timestamp":"2026-04-03T13:08:10.161454+00:00","status":"success","summary":"compute_graph_prune completed successfully.","started_at":"2026-04-03T13:08:10Z","finished_at":"2026-04-03T13:08:10Z","duration_ms":951,"rows_seen":0,"rows_processed":0,"rows_written":1,"rows_skipped":0,"rows_failed":0,"batches_completed":0,"checkpoint_before":null,"checkpoint_after":null,"has_more":false,"error_text":null,"error":null,"warnings":[],"meta":{"job_key":"compute_graph_prune","schema_version":"job_result.v2","nodes_created":0,"nodes_merged":0,"relationships_created":0,"relationships_merged":0,"timestamp":"2026-04-03T13:08:10.161223+00:00","deleted_relationships":0,"warnings":[]}}
{"event":"graph.prune.completed","timestamp":"2026-04-03T13:08:19.704362+00:00","status":"success","summary":"compute_graph_prune completed successfully.","started_at":"2026-04-03T13:08:19Z","finished_at":"2026-04-03T13:08:19Z","duration_ms":958,"rows_seen":0,"rows_processed":0,"rows_written":1,"rows_skipped":0,"rows_failed":0,"batches_completed":0,"checkpoint_before":null,"checkpoint_after":null,"has_more":false,"error_text":null,"error":null,"warnings":[],"meta":{"job_key":"compute_graph_prune","schema_version":"job_result.v2","nodes_created":0,"nodes_merged":0,"relationships_created":0,"relationships_merged":0,"timestamp":"2026-04-03T13:08:19.689389+00:00","deleted_relationships":0,"warnings":[]}}
{"event":"graph.topology.completed","timestamp":"2026-04-03T13:08:34.529879+00:00","status":"success","summary":"compute_graph_topology_metrics completed successfully.","started_at":"2026-04-03T13:08:34Z","finished_at":"2026-04-03T13:08:34Z","duration_ms":29227,"rows_seen":204651,"rows_processed":204651,"rows_written":204651,"rows_skipped":0,"rows_failed":0,"batches_completed":0,"checkpoint_before":null,"checkpoint_after":null,"has_more":false,"error_text":null,"error":null,"warnings":[],"meta":{"job_key":"compute_graph_topology_metrics","schema_version":"job_result.v2","nodes_created":0,"nodes_merged":0,"relationships_created":0,"relationships_merged":0,"timestamp":"2026-04-03T13:08:34.529121+00:00"}}
{"event":"graph.topology.completed","timestamp":"2026-04-03T13:08:34.676922+00:00","status":"success","summary":"compute_graph_topology_metrics completed successfully.","started_at":"2026-04-03T13:08:34Z","finished_at":"2026-04-03T13:08:34Z","duration_ms":24511,"rows_seen":204651,"rows_processed":204651,"rows_written":204651,"rows_skipped":0,"rows_failed":0,"batches_completed":0,"checkpoint_before":null,"checkpoint_after":null,"has_more":false,"error_text":null,"error":null,"warnings":[],"meta":{"job_key":"compute_graph_topology_metrics","schema_version":"job_result.v2","nodes_created":0,"nodes_merged":0,"relationships_created":0,"relationships_merged":0,"timestamp":"2026-04-03T13:08:34.675509+00:00"}}

zkill.log 60 MB

modified 2s ago

{"ts":"2026-04-03T13:09:29.742792+00:00","level":"info","logger":"supplycore.zkill","message":"zkill.sequence_fetch","event":"zkill.sequence_fetch","job_key":"killmail_r2z2_sync","sequence_id":96740822,"http_status":200,"payload_classification":"valid_payload","fetch_outcome":"valid_http_200","valid_non_actionable":false}
{"ts":"2026-04-03T13:09:31.497847+00:00","level":"info","logger":"supplycore.zkill","message":"zkill.batch_completed","event":"zkill.batch_completed","job_key":"killmail_r2z2_sync","batch_index":64,"rows_seen":1,"rows_matched":0,"rows_filtered_out":0,"rows_skipped_existing":0,"rows_write_attempted":1,"rows_written":1,"rows_failed":0,"cursor_after":"96740822","first_sequence_seen":96740822,"last_sequence_seen":96740822,"reason_for_zero_write":"","checkpoint_state":{"checkpoint_updated":true,"cursor":"96740822","reason":"cursor_checkpointed"},"duration_ms":171514,"memory_usage_bytes":33030144,"running_rows_seen":64,"running_rows_written":64}
{"ts":"2026-04-03T13:09:31.497974+00:00","level":"info","logger":"supplycore.zkill","message":"zkill.sequence_processed","event":"zkill.sequence_processed","job_key":"killmail_r2z2_sync","sequence_id":96740822,"http_status":200,"payload_classification":"valid_payload","processing_outcome":"inserted","rows_matched":0,"rows_filtered_out":0,"rows_skipped_existing":0,"rows_written":1,"rows_failed":0,"valid_non_actionable":false,"checkpoint_state":{"checkpoint_updated":true,"cursor":"96740822","reason":"cursor_checkpointed"}}
{"ts":"2026-04-03T13:09:31.657276+00:00","level":"info","logger":"supplycore.zkill","message":"zkill.sequence_fetch","event":"zkill.sequence_fetch","job_key":"killmail_r2z2_sync","sequence_id":96740823,"http_status":404,"payload_classification":"not_found","fetch_outcome":"sleep_and_retry","valid_non_actionable":true}
{"ts":"2026-04-03T13:09:41.808364+00:00","level":"info","logger":"supplycore.zkill","message":"zkill.sequence_fetch","event":"zkill.sequence_fetch","job_key":"killmail_r2z2_sync","sequence_id":96740823,"http_status":404,"payload_classification":"not_found","fetch_outcome":"sleep_and_retry","valid_non_actionable":true}

influx-rollup-export.log 14.4 MB

modified 3m ago

{"ts":"2026-04-03T13:05:55.970322+00:00","level":"info","logger":"supplycore.influx_export","message":"Completed InfluxDB rollup export dataset.","dataset":"doctrine_group_activity_1d","source_rows":30,"written_rows":30,"cursor_end":"2026-04-01 15:20:05","dry_run":false}
{"ts":"2026-04-03T13:05:55.971829+00:00","level":"info","logger":"supplycore.influx_export","message":"Starting InfluxDB rollup export dataset.","dataset":"doctrine_fit_stock_pressure_1d","table":"doctrine_fit_stock_pressure_1d","run_mode":"incremental","cursor_start":"2026-04-01 09:20:05","dry_run":false}
{"ts":"2026-04-03T13:05:55.977440+00:00","level":"info","logger":"supplycore.influx_export","message":"Exported InfluxDB rollup batch.","dataset":"doctrine_fit_stock_pressure_1d","batch_rows":114,"source_rows_total":114,"written_rows_total":114,"cursor_end":"2026-04-01 15:20:05","dry_run":false}
{"ts":"2026-04-03T13:05:55.978489+00:00","level":"info","logger":"supplycore.influx_export","message":"Completed InfluxDB rollup export dataset.","dataset":"doctrine_fit_stock_pressure_1d","source_rows":114,"written_rows":114,"cursor_end":"2026-04-01 15:20:05","dry_run":false}
{"ts":"2026-04-03T13:05:55.978593+00:00","level":"info","logger":"supplycore.influx_export","message":"InfluxDB rollup export completed.","dataset_count":11,"datasets":[{"dataset":"market_item_price_1h","run_status":"success","source_rows":120258,"written_rows":120258,"cursor_end":"2026-04-03 14:56:37"},{"dataset":"market_item_price_1d","run_status":"success","source_rows":14062,"written_rows":14062,"cursor_end":"2026-04-03 14:57:04"},{"dataset":"market_item_stock_1h","run_status":"success","source_rows":120258,"written_rows":120258,"cursor_end":"2026-04-03 14:56:29"},{"dataset":"market_item_stock_1d","run_status":"success","source_rows":14062,"written_rows":14062,"cursor_end":"2026-04-03 14:56:58"},{"dataset":"killmail_item_loss_1h","run_status":"success","source_rows":0,"written_rows":0,"cursor_end":"2026-01-26 02:30:12"},{"dataset":"killmail_item_loss_1d","run_status":"success","source_rows":0,"written_rows":0,"cursor_end":"2026-01-26 02:30:12"},{"dataset":"killmail_hull_loss_1d","run_status":"success","source_rows":0,"written_rows":0,"cursor_end":"2026-01-26 02:30:12"},{"dataset":"killmail_doctrine_activity_1d","run_status":"success","source_rows":0,"written_rows":0,"cursor_end":"2026-01-26 02:30:12"},{"dataset":"doctrine_fit_activity_1d","run_status":"success","source_rows":113,"written_rows":113,"cursor_end":"2026-04-03 14:56:29"},{"dataset":"doctrine_group_activity_1d","run_status":"success","source_rows":30,"written_rows":30,"cursor_end":"2026-04-01 15:20:05"},{"dataset":"doctrine_fit_stock_pressure_1d","run_status":"success","source_rows":114,"written_rows":114,"cursor_end":"2026-04-01 15:20:05"}],"dry_run":false}

backfill-runner.log 1.3 MB

modified 57s ago

{"ts":"2026-04-03T13:04:44.003173+00:00","level":"info","logger":"supplycore.backfill_runner","message":"Backfill runner cycle completed","event":"backfill_runner.cycle_completed","worker_id":"eve-intel-server-1646307","cycle":93,"cycle_duration_s":0.3,"status":"success","start_date":"2026-01-01","up_to_date":false,"months_processed":1,"written":0}
{"ts":"2026-04-03T13:05:44.389804+00:00","level":"info","logger":"supplycore.backfill_runner","message":"Backfill runner cycle completed","event":"backfill_runner.cycle_completed","worker_id":"eve-intel-server-1646307","cycle":94,"cycle_duration_s":0.4,"status":"success","start_date":"2026-01-01","up_to_date":false,"months_processed":1,"written":0}
{"ts":"2026-04-03T13:06:44.734219+00:00","level":"info","logger":"supplycore.backfill_runner","message":"Backfill runner cycle completed","event":"backfill_runner.cycle_completed","worker_id":"eve-intel-server-1646307","cycle":95,"cycle_duration_s":0.3,"status":"success","start_date":"2026-01-01","up_to_date":false,"months_processed":1,"written":0}
{"ts":"2026-04-03T13:07:45.071638+00:00","level":"info","logger":"supplycore.backfill_runner","message":"Backfill runner cycle completed","event":"backfill_runner.cycle_completed","worker_id":"eve-intel-server-1646307","cycle":96,"cycle_duration_s":0.3,"status":"success","start_date":"2026-01-01","up_to_date":false,"months_processed":1,"written":0}
{"ts":"2026-04-03T13:08:45.425115+00:00","level":"info","logger":"supplycore.backfill_runner","message":"Backfill runner cycle completed","event":"backfill_runner.cycle_completed","worker_id":"eve-intel-server-1646307","cycle":97,"cycle_duration_s":0.4,"status":"success","start_date":"2026-01-01","up_to_date":false,"months_processed":1,"written":0}

evewho-runner.log 564.3 KB

modified 3m ago

{"ts":"2026-04-03T13:05:36.900459+00:00","level":"info","logger":"supplycore.evewho_runner","message":"Starting enrichment sync cycle"}
{"ts":"2026-04-03T13:05:58.827740+00:00","level":"info","logger":"supplycore.evewho_runner","message":"Enrichment sync completed","event":"evewho_runner.enrichment_sync_done","status":"success","summary":"Enriched 50/50 characters in 5 batch(es), 21926ms"}
{"ts":"2026-04-03T13:05:58.828150+00:00","level":"info","logger":"supplycore.evewho_runner","message":"EveWho runner cycle completed","event":"evewho_runner.cycle_completed","worker_id":"eve-intel-server-1646312","cycle":9,"cycle_duration_s":595.3,"status":"success"}
{"ts":"2026-04-03T13:05:58.828231+00:00","level":"info","logger":"supplycore.evewho_runner","message":"Sleeping 30 seconds before next cycle","event":"evewho_runner.sleeping","loop_sleep":30}
{"ts":"2026-04-03T13:06:28.828490+00:00","level":"info","logger":"supplycore.evewho_runner","message":"Starting alliance member sync cycle"}

cron.log 0 B

modified 2d ago

killmail-backfill.log 0 B

modified 2d ago

Recent Runs

Job	Status	Last run	Duration	Written / Read	Recent OK	Summary / Error
scheduler.job.compute_battle_rollups	Success	37s ago	0s	25,971 / 13,604	2 runs	Rolled up 13604 killmails into battles across 14 batches.
scheduler.job.compute_graph_topology_metrics	Success	1m ago	0s	204,651 / 204,651	11 runs	compute_graph_topology_metrics completed successfully.
scheduler.job.compute_graph_sync	Success	1m ago	0s	207,342 / 207,341	2 runs	Graph sync pipeline (sync+derived+prune+topology) finished with status success.
scheduler.job.graph_universe_sync	Success	1m ago	0s	23,756 / 23,756	10 runs	Synced 114 regions, 1184 constellations, 8490 systems, 13968 stargates.
scheduler.job.market_hub_current_sync	Success	1m ago	0s	32,739 / 33,550	2 runs	Ingested/projected market-hub orders (32739 writes across 1 sources).
scheduler.job.analytics_bucket_1d_sync	Success	1m ago	0s	0 / 7,442,476	10 runs	Upserted 0 stock and 0 price daily bucket rows.
scheduler.job.analytics_bucket_1h_sync	Success	1m ago	0s	31,926 / 4,582,633	2 runs	Upserted 15963 stock and 15963 price hourly bucket rows.
scheduler.job.alliance_historical_sync	Success	1m ago	0s	0 / 5,256	10 runs	Materialized 0 alliance_structure historical snapshot rows.
scheduler.job.market_hub_historical_sync	Success	1m ago	0s	0 / 10,376	10 runs	Materialized 0 market_hub historical snapshot rows.
scheduler.job.esi_alliance_history_sync	Success	1m ago	0s	0 / 5	10 runs	Fetched 5 characters, wrote 0 alliance history periods (0 errors, 0 skipped).
scheduler.job.graph_analyst_recalibration	Success	1m ago	0s	0 / 0	10 runs	Insufficient labels for recalibration (0/10).
scheduler.job.graph_model_audit	Success	1m ago	0s	0 / 0	10 runs	graph_model_audit completed with status success.
scheduler.job.compute_graph_prune	Success	1m ago	0s	1 / 0	10 runs	compute_graph_prune completed successfully.
scheduler.job.killmail_zkb_repair	Success	1m ago	0s	0 / 0	10 runs	killmail_zkb_repair completed with status success.
scheduler.job.shell_corp_detection	Success	1m ago	0s	2,689 / 2,745	10 runs	shell_corp_detection completed with status success.
scheduler.job.cache_expiry_cleanup_sync	Success	1m ago	0s	0 / 0	10 runs	Deleted 0 expired cache rows (adj snapshots: 0).
scheduler.job.intelligence_pipeline	Success	1m ago	0s	0 / 204,628	10 runs	Intelligence pipeline complete: 0 suspicion signals, 0 overlap records exported.
scheduler.job.entity_metadata_resolve_sync	Success	1m ago	0s	500 / 500	2 runs	Resolved 500/500 pending entities (0 failed, 0 still pending).
scheduler.job.esi_character_queue_sync	Success	1m ago	0s	5 / 4,327	10 runs	Queued 5 new attacker character IDs for ESI lookup (5 pending, 4322 done).
scheduler.job.rebuild_ai_briefings	Success	1m ago	0s	228 / 114	10 runs	Rebuilt 228 doctrine AI briefings from deterministic fit activity inputs.
scheduler.job.graph_temporal_metrics_sync	Success	1m ago	0s	72 / 72	10 runs	Temporal metrics computed for 3 windows, 72 rows written, 0 drift flags.
scheduler.job.graph_typed_interactions_sync	Success	1m ago	0s	0 / 0	10 runs	Typed interactions: 0 Neo4j rels created, 0 MariaDB rows written.
scheduler.job.escalation_detection	Success	1m ago	0s	2,275 / 2,651	10 runs	escalation_detection completed with status success.
scheduler.job.forecasting_ai_sync	Success	1m ago	0s	2 / 300	10 runs	Refreshed forecasting snapshot with 300 candidates.
scheduler.job.compute_graph_derived_relationships	Success	1m ago	0s	868 / 868	10 runs	compute_graph_derived_relationships completed successfully.
scheduler.job.compute_suspicion_scores	Success	1m ago	0s	18,166 / 19,471	10 runs	Scored 9083 characters (8197 filtered below sample threshold).
scheduler.job.compute_behavioral_baselines	Success	1m ago	0s	9,083 / 19,471	10 runs	Computed behavioral baselines for 9083 characters from 19471 actor features.
scheduler.job.corp_standings_sync	Success	1m ago	0s	0 / 0	10 runs	Skipped — no ESI OAuth token available.
scheduler.job.compute_graph_sync_battle_intelligence	Success	1m ago	0s	0 / 0	10 runs	compute_graph_sync_battle_intelligence completed successfully.
scheduler.job.compute_graph_sync_doctrine_dependency	Success	1m ago	0s	1,822 / 1,822	10 runs	compute_graph_sync_doctrine_dependency completed successfully.
scheduler.job.compute_alliance_relationships	Success	1m ago	0s	0 / 0	10 runs	No new killmails to process for alliance relationships.
scheduler.job.battle_type_classification	Success	1m ago	0s	0 / 0	10 runs	battle_type_classification completed with status success.
scheduler.job.current_state_refresh_sync	Success	1m ago	0s	134 / 67	2 runs	Refreshed scheduler current-state rows for 134 jobs.
scheduler.job.activity_priority_summary_sync	Success	1m ago	0s	114 / 114	2 runs	Refreshed doctrine activity snapshots with 114 upserts.
scheduler.job.dashboard_summary_sync	Success	1m ago	0s	2 / 5,782	2 runs	Refreshed dashboard summary intelligence snapshot.
scheduler.job.alliance_current_sync	Success	1m ago	0s	0 / 20,000	2 runs	Ingested/projected alliance structure orders (0 writes across 1 structures).
scheduler.job.loss_demand_summary_sync	Success	1m ago	0s	2 / 0	2 runs	Refreshed loss-demand intelligence snapshot with 0 rows.
scheduler.job.doctrine_intelligence_sync	Success	1m ago	0s	3,964 / 114	2 runs	Updated doctrine intelligence stock/activity rows (3964 upserts).
scheduler.job.compute_suspicion_scores_v2	Success	1m ago	0s	8,959 / 8,959	2 runs	Scored 8959 characters from 8959 behavioral baselines.
scheduler.job.compute_counterintel_pipeline	Success	1m ago	0s	0 / 0	2 runs	Processed 0 eligible 100+ participant battles across 0 batches.
scheduler.job.theater_analysis	Success	2m ago	0s	32,183 / 609	2 runs	Analyzed 609 theaters.
scheduler.job.compute_battle_actor_features	Success	2m ago	0s	19,471 / 19,471	2 runs	Extracted actor features for 19471 participants, wrote 19471 rows.
scheduler.job.graph_data_quality_check	Success	2m ago	0s	1 / 204,550	2 runs	Quality gate passed: score 0.6996 (204550 characters, 279 orphans, 1034815 dups).
scheduler.job.temporal_behavior_detection	Success	2m ago	0s	3,852 / 23,524	10 runs	Temporal behavior detection complete: 23524 characters analysed, 3852 evidence rows written, 0 Neo4j tags.
scheduler.job.compute_graph_insights	Success	2m ago	0s	26,828 / 25,839	2 runs	compute_graph_insights completed successfully.
scheduler.job.compute_buy_all	Success	2m ago	0s	480 / 250	2 runs	Precomputed 4 buy-all request(s) with 480 item rows.
scheduler.job.graph_motif_detection_sync	Success	2m ago	0s	802 / 802	10 runs	Motif detection: 802 motifs found (triangle=500, star=2, chain=300, fleet_core=0, rotating_scout=0).
scheduler.job.compute_copresence_edges	Success	2m ago	0s	0 / 0	10 runs	No characters to process.
scheduler.job.compute_signals	Success	2m ago	0s	120 / 120	2 runs	Generated 120 intelligence signals from 120 buy-all items.
scheduler.job.compute_graph_sync_killmail_entities	Success	2m ago	0s	284 / 284	2 runs	Projected 284 killmails into Neo4j.
scheduler.job.market_comparison_summary_sync	Success	2m ago	0s	4,113 / 11,099	2 runs	Market comparison: 4113 items evaluated (2617 in both markets).
scheduler.job.tracked_alliance_member_sync	Success	2m ago	0s	89,547 / 0	2 runs	Tracked alliance sync \| 0 alliances, 0 corps \| 89497 members graphed, 50 ESI-enriched \| 100/300 ESI, 0/30 EveWho, 33569ms
scheduler.job.graph_community_detection_sync	Success	2m ago	0s	204,550 / 204,550	10 runs	Community detection: 204550 characters assigned to 201668 communities, 2110 bridges.
scheduler.job.deal_alerts_sync	Success	2m ago	0s	7,204 / 15,632	2 runs	Materialized 7204 active deal alerts.
scheduler.job.compute_battle_anomalies	Success	2m ago	0s	7,018 / 3,509	2 runs	Detected 3509 anomalies (84 high-sustain) from 3509 rows.
scheduler.job.compute_battle_target_metrics	Success	2m ago	0s	1,258 / 633	2 runs	Scored 633 target metrics across 1 batches (4 unscored).
scheduler.job.compute_alliance_dossiers	Success	2m ago	0s	772 / 386	10 runs	Computed 386 alliance dossiers.
scheduler.job.compute_cohort_baselines	Success	2m ago	0s	24,979 / 8,959	10 runs	Assigned 15975 cohort memberships across 5 cohorts for 8959 characters. Computed 45 feature baselines. Updated 8959 cohort-relative scores.
scheduler.job.compute_threat_corridors	Success	2m ago	0s	3,920 / 2,803	10 runs	Found 200 threat corridors across 2803 active systems.
scheduler.job.compute_graph_sync_killmail_edges	Success	2m ago	0s	4 / 4	10 runs	Synced 4 killmail edges (ATTACKED_ON + VICTIM_OF) into Neo4j.
scheduler.job.staging_system_detection	Success	2m ago	0s	1,481 / 1,072	10 runs	staging_system_detection completed with status success.
scheduler.job.pre_op_join_detection	Success	2m ago	0s	4 / 4	10 runs	pre_op_join_detection completed with status success.
scheduler.job.theater_suspicion	Success	2m ago	0s	600 / 609	9 runs	Integrated suspicion for 600 theaters.
scheduler.job.compute_character_feature_windows	Success	2m ago	0s	0 / 0	10 runs	Computed feature windows across 0 batches, wrote 0 feature rows.
scheduler.job.graph_evidence_paths_sync	Success	2m ago	0s	0 / 0	10 runs	No flagged characters above threshold.
scheduler.job.jump_bridge_sync	Success	2m ago	0s	0 / 0	10 runs	jump_bridge_sync completed with status success.
scheduler.job.compute_economic_warfare	Success	2m ago	0s	17,854 / 1,822	10 runs	Economic warfare: 1273 fit families, 1421 modules scored.
scheduler.job.theater_graph_integration	Success	2m ago	0s	17,550 / 609	9 runs	Enriched 600 theaters with graph data.
scheduler.job.evewho_enrichment_sync	Success	2m ago	0s	50 / 50	2 runs	Enriched 50/50 characters in 5 batch(es), 25036ms
killmail.r2z2.stream	Running	3m ago	2h	0 / 0	2 runs
scheduler.job.evewho_alliance_member_sync	Success	3m ago	0s	58,811 / 2,863	2 runs	Graph crawl: Phase 1 \| 1 alliances, 33 corps swept \| 58811 members graphed, 2863 queued \| 0 chars enriched, 0c/0a discovered \| 100/100 API calls, 589026ms
influx.rollup_export.market_item_stock_1d	Success	3m ago	0s	14,062 / 14,062	-
influx.rollup_export.killmail_item_loss_1h	Success	3m ago	0s	0 / 0	-
influx.rollup_export.killmail_item_loss_1d	Success	3m ago	0s	0 / 0	-
influx.rollup_export.killmail_hull_loss_1d	Success	3m ago	0s	0 / 0	-
influx.rollup_export.killmail_doctrine_activity_1d	Success	3m ago	0s	0 / 0	-
influx.rollup_export.doctrine_fit_activity_1d	Success	3m ago	0s	113 / 113	-
influx.rollup_export.doctrine_group_activity_1d	Success	3m ago	0s	30 / 30	-
influx.rollup_export.doctrine_fit_stock_pressure_1d	Success	3m ago	0s	114 / 114	-
influx.rollup_export.market_item_stock_1h	Success	3m ago	4s	120,258 / 120,258	-