Data Science Functions Reference ⚡

Functions Overview

Navigation

Use the Index below to jump to the details of a function.

🎯 Recommendations & Scoring

Function	Purpose
`recommend`	End-to-end ProcessAI recommendation pipeline (candidate generation → scoring → ranking).
`candidate_score`	Score candidate settings based on user-interest metrics, quality tests, constraints, and duration.
`compute_recommendation_adherence`	Adherence score of active recommendations vs. actuals over time buckets.
`recommendation_achievement`	Per-period “achievement” score for a recommendation.

🧠 Modeling & Inference

Function	Purpose
`infer_from_model`	Run inference from a production ML/optimization model given metric-group inputs.
`infer_from_oql`	Run inference from any OQL-shaped inputs (optionally return probabilities).
`compute_regression_slos`	Compute regression SLOs (Correlation, R², MAPE, MSE, Nsamples) with group & cross-group rollups.
`linear_polyfit`	Fit degree-1 regression; return slope or status vs. thresholds.

🧩 Segmentation & Clustering

Function	Purpose
`threshold_segmentation`	Segment based on thresholds and optional interval changes.
`detect_ramp_up_from_downtime`	Detect ramp-up windows after downtime using learned thresholds.
`dummy_cluster`	Assign dummy cluster indices to rows.
`segment_repeatability`	Pairwise segment repeatability from process & outcome distances.
`discover_motifs`	Find time-series motifs via matrix profile (experimental).
`mpsearch`	Matrix profile search over a time window (experimental).

📏 Distance & Similarity

Function	Purpose
`compute_distance`	Distance to a reference vector with optional sort & column management.

📊 Aggregation & Post-processing

Function	Purpose
`final_aggregate`	Final pass aggregations over OQL results via JSON specs.
`accumulate_cost_savings`	Accumulate cost, projected cost, and savings from a cost-rate series.
`recommend_performance_recipe`	Surface top-performing stable segment’s “recipe” and opportunity vs. target.

Function Details

`• recommend`

End-to-end recommendation workflow for ProcessAI (v0.1 alpha): candidate generation from stable periods, scoring (supports recsys and predictive_quality), optional penalties, and ranking.

Arguments

Name	Type	Description
`input_data`	OqlData	Input table containing current/stable metrics, specs, tests, and any penalty inputs.
`action_metric_group_name`	list[str]	Aliases of actionable metric groups to swap.
`action_metric_group_uuid`	list[str]	UUIDs of actionable metric groups.
`quality_test_identifiers`	list[str]	Names/aliases of quality tests.
`model_identifiers`	list[str]	Model identifiers (MLFlow) per quality test.
`score_normalize`	list[str]	Per-test normalization (e.g., `"as-is"`, `"normalize"`, `"log-normalize"`).
`score_aggregate`	str	Score aggregation method (e.g., `"additive"`).
`candidate_index_cols`	list[str]	Columns that uniquely index candidate: must include `id`, `start`, `end`.
`infer_metric_group_name`	list[str] \| None	Extra metric groups to feed into inference models (aliases).
`infer_metric_group_uuid`	list[str] \| None	UUIDs matching `infer_metric_group_name`.
`current_prefix`	str \| None	Prefix for current metrics in `input_data`.
`candidate_prefix`	str \| None	Prefix for stable/candidate metrics in `input_data`.
`quality_spec_lsl_postfix`	str \| None	Postfix for LSL columns.
`quality_spec_usl_postfix`	str \| None	Postfix for USL columns.
`quality_spec_target_postfix`	str \| None	Postfix for Target columns.
`penalty_col`	str \| None	Column with multiplicative penalty factor.
`score_col`	str \| None	Name of final score column.
`keep_cols`	list[str] \| None	Columns to pass through to the result.
`verbose`	str \| None	`"full"`, `"standard"`, or `"silent"`.
`rename_cols`	str \| None	Dict-in-string to rename output columns.
`first_cols`	list[str] \| None	Columns to pin first in result order.
`local_candidate_stepsizes`	str \| None	JSON dict of local search steps around current conditions.
`non_quality_col`	str \| None	Column for non-quality objective.
`quality_score_multiplier_col`	str \| None	Column to scale quality score.

Example

WITH current_metric () AS (
  AGGREGATE
    avg(metric_group('<ACTION_MG_UUID_1>')) AS action_1,
    avg(metric_group('<ACTION_MG_UUID_2>')) AS action_2,
    avg(metric_group('<ACTION_MG_UUID_3>')) AS action_3,
    avg(metric_group('<INFER_MG_UUID_1>')) AS infer_1,
    avg(metric_group('<INFER_MG_UUID_2>')) AS infer_2,
    avg(metric_group('<INFER_MG_UUID_3>')) AS infer_3,
    avg(metric_group('<INFER_MG_UUID_4>')) AS infer_4
  WHERE time FROM now()-1m TO now()
    AND line.id = '<LINE_ID_1>'
  SETTINGS backend=v1
),
current_test () AS (
  AGGREGATE
    lastIf(offline_quality.results.result, offline_quality.results.testid = '<QUALITY_TEST_NAME_1>') AS quality_test_1,
    lastIf(offline_quality.results.result, offline_quality.results.testid = '<QUALITY_TEST_NAME_2>') AS quality_test_2,
    lastIf(offline_quality.results.result, offline_quality.results.testid = '<QUALITY_TEST_NAME_3>') AS quality_test_3
  WHERE time FROM now()-2h TO now()
    AND line.id = '<LINE_ID_1>'
  SETTINGS backend=v1
),
stable () AS (
  AGGREGATE
    interval.stable_period.start AS start,
    interval.stable_period.end   AS end,
    avg(metric_group('<ACTION_MG_UUID_1>')) AS action_1,
    avg(metric_group('<ACTION_MG_UUID_2>')) AS action_2,
    avg(metric_group('<ACTION_MG_UUID_3>')) AS action_3,
    coalesce(avg(interval.run.target('<QUALITY_MG_UUID_1>').lsl),0) AS quality_test_1_lsl,
    avg(interval.run.target('<QUALITY_MG_UUID_1>').usl)             AS quality_test_1_usl,
    coalesce(avg(interval.run.target('<QUALITY_MG_UUID_2>').lsl),0) AS quality_test_2_lsl,
    avg(interval.run.target('<QUALITY_MG_UUID_3>').usl)             AS quality_test_3_usl,
    avg(interval.run.target('<QUALITY_MG_UUID_1>').value)           AS quality_test_1_target,
    avg(interval.run.target('<QUALITY_MG_UUID_2>').value)           AS quality_test_2_target,
    avg(interval.run.target('<QUALITY_MG_UUID_3>').value)           AS quality_test_3_target,
    avg(metric_group('<REF_MG_UUID_1>')) AS ref_1,
    avg(metric_group('<REF_MG_UUID_2>')) AS ref_2,
    avg(metric_group('<REF_MG_UUID_3>')) AS ref_3,
    avg(metric_group('<NON_QUALITY_MG_UUID_1>')) AS non_quality_1,
    avg(metric_group('<NON_QUALITY_MG_UUID_2>')) AS non_quality_2
  WHERE time FROM date_trunc(now()-7M, 'D') TO date_trunc(now(), 'D')
    AND line.id    = '<LINE_ID_1>'
    AND product.id = '<PRODUCT_ID_1>'
    AND interval.state.type = 'Uptime'
    AND interval.stable_period.value != ''
  GROUP BY interval.stable_period AS id
  SETTINGS backend=v1
)
AGGREGATE
  stable.id   AS id,
  stable.start AS start,
  stable.end   AS end,
  current_metric.action_1 AS current_action_1,
  current_metric.action_2 AS current_action_2,
  current_metric.action_3 AS current_action_3,
  stable.action_1   AS stable_action_1,
  stable.action_2   AS stable_action_2,
  stable.action_3   AS stable_action_3,
  current_metric.infer_1 AS current_infer_1,
  current_metric.infer_2 AS current_infer_2,
  current_metric.infer_3 AS current_infer_3,
  current_metric.infer_4 AS current_infer_4,
  stable.quality_test_1_lsl AS quality_test_1_lsl,
  stable.quality_test_1_usl AS quality_test_1_usl,
  stable.quality_test_2_lsl AS quality_test_2_lsl,
  stable.quality_test_3_usl AS quality_test_3_usl,
  stable.quality_test_1_target AS quality_test_1_target,
  stable.quality_test_2_target AS quality_test_2_target,
  stable.quality_test_3_target AS quality_test_3_target,
  current_test.quality_test_1  AS quality_test_1,
  current_test.quality_test_2  AS quality_test_2,
  current_test.quality_test_3  AS quality_test_3,
  greatest(1 - abs(current_action_1 - stable_action_1) / greatest(current_action_1, 1e-9), 0) *
  greatest(1 - abs(current_action_2 - stable_action_2) / greatest(current_action_2, 1e-9), 0) *
  greatest(1 - abs(current_action_3 - stable_action_3) / greatest(current_action_3, 1e-9), 0) AS penalty_factor,
  (stable.ref_1 * stable.ref_2 + stable.ref_3) AS quality_reference,
  (stable.non_quality_1 * stable.non_quality_2) AS non_quality_col,
  0.5 AS quality_score_multiplier_col
JOIN stable        USING ()
JOIN current_metric USING ()
JOIN current_test   USING ()
WHERE TIME FROM now()-6M TO now()
  AND line.id = '<LINE_ID_1>'
HAVING quality_reference > 0
APPLY honk2science recommend(
  action_metric_group_name = ['action_1','action_2','action_3'],
  action_metric_group_uuid = ['<ACTION_MG_UUID_1>','<ACTION_MG_UUID_2>','<ACTION_MG_UUID_3>'],
  quality_test_identifiers = ['quality_test_1','quality_test_2','quality_test_3'],
  model_identifiers        = ['<MODEL_IDENTIFIER_1>','<MODEL_IDENTIFIER_2>','<MODEL_IDENTIFIER_3>'],
  score_normalize          = ['as-is','normalize','log-normalize'],
  score_aggregate          = 'additive',
  candidate_index_cols     = ['id','start','end'],
  infer_metric_group_name  = ['infer_1','infer_2','infer_3'],
  infer_metric_group_uuid  = ['<INFER_MG_UUID_1>','<INFER_MG_UUID_2>','<INFER_MG_UUID_3>'],
  current_prefix           = 'current_',
  candidate_prefix         = 'stable_',
  quality_spec_lsl_postfix    = '_lsl',
  quality_spec_usl_postfix    = '_usl',
  quality_spec_target_postfix = '_target',
  penalty_col                 = 'penalty_factor',
  score_col                   = 'rank_score',
  keep_cols                   = ['quality_reference','stable_action_1','current_action_1','penalty_factor'],
  verbose                     = 'silent',
  rename_cols                 = "{'quality_reference':'my_first_name','current_action_1':'my_second_name'}",
  first_cols                  = ['stable_action_1','penalty_factor'],
  local_candidate_stepsizes   = '{"infer_1":{"step_size":0.1,"max":10,"min":0},"infer_2":{"step_size":"<STEP_COL_NAME>","max":"<MAX_COL_NAME>","min":"<MIN_COL_NAME>"},"infer_3":{"step_size":0.1,"max":10,"min":0}}',
  non_quality_col             = 'non_quality_col',
  quality_score_multiplier_col= 'quality_score_multiplier_col'
)
SETTINGS backend=v1;

Returns OqlData with score_col plus candidate indexing columns (id, start, end) and any keep_cols.

Raises GRPCError – On invalid or missing inputs/models/scoring setup.

`• candidate_score`

Compute a per-candidate score using user-interest metric deltas, quality tests (online/offline or predicted), and constraint measures; optionally weight by candidate duration.

Arguments

Name	Type	Description
`input_data`	OqlData	Candidate rows with current & stable values, specs, tests.
`user_interest_cols`	list[str]	Column aliases the user cares to change.
`user_interest_metric_groups`	list[str]	1:1 metric-group UUIDs for `user_interest_cols` (order matters).
`quality_test_identifiers`	list[str]	Names of quality tests in `input_data`.
`model_identifiers`	list[str]	MLFlow model identifiers per quality test (order matters).
`constraint_measure_cols`	list[str]	Constraint columns (e.g., moisture, freeness).
`constraint_measure_scaling_factors`	list[float]	Scaling factors per constraint column.
`candidate_duration_col`	str	Column with candidate duration (or adjusted duration).
`candidate_score_col`	str	Output score column name (default: `score`).
`remove_cols`	list[str] \| None	Columns to drop before returning.

Example

WITH current_metric () AS (
  AGGREGATE
    avg(metric_group('<USER_INTEREST_MG_UUID_1>')) AS draw,
    avg(metric_group('<USER_INTEREST_MG_UUID_2>')) AS steam,
    avg(metric_group('<USER_INTEREST_MG_UUID_3>')) AS hpdt,
    avg(metric_group('<CONSTRAINT_MG_UUID_1>')) AS moisture,
    avg(metric_group('<CONSTRAINT_MG_UUID_2>')) AS freeness,
    avg(metric_group('<CONSTRAINT_MG_UUID_3>')) AS speed,
    avg(metric_group('<PQ_MG_UUID_1>')) AS predictive_quality_test_1,
    avg(metric_group('<PQ_MG_UUID_2>')) AS predictive_quality_test_2
  WHERE time FROM now()-1m TO now()
    AND line.id = '<LINE_ID_1>'
  SETTINGS backend=v1
),
current_test () AS (
  AGGREGATE
    abs(lastIf(offline_quality.results.minvalue, offline_quality.results.testid = '<QUALITY_TEST_NAME_1>')) AS current_quality_test_1,
    abs(lastIf(offline_quality.results.minvalue, offline_quality.results.testid = '<QUALITY_TEST_NAME_2>')) AS current_quality_test_2
  WHERE time FROM now()-2h TO now()
    AND line.id = '<LINE_ID_1>'
  SETTINGS backend=v1
)
AGGREGATE
  interval.stable_period.start AS start,
  interval.stable_period.end   AS end,
  avg(metric_group('<USER_INTEREST_MG_UUID_1>')) AS stable_draw,
  avg(metric_group('<USER_INTEREST_MG_UUID_2>')) AS stable_steam,
  avg(metric_group('<USER_INTEREST_MG_UUID_3>')) AS stable_hpdt,
  avg(metric_group('<CONSTRAINT_MG_UUID_1>')) AS stable_moisture,
  avg(metric_group('<CONSTRAINT_MG_UUID_2>')) AS stable_freeness,
  avg(metric_group('<CONSTRAINT_MG_UUID_3>')) AS stable_speed,
  coalesce(avg(interval.run.target('<PQ_MG_UUID_1>').lsl),0) AS quality_test_1_lsl,
  avg(interval.run.target('<PQ_MG_UUID_1>').value) AS quality_test_1_target,
  coalesce(avg(interval.run.target('<PQ_MG_UUID_2>').lsl),0) AS quality_test_2_lsl,
  avg(interval.run.target('<PQ_MG_UUID_2>').usl) AS quality_test_2_usl,
  avg(interval.run.target('<PQ_MG_UUID_2>').value) AS quality_test_2_target,
  timeinif() AS duration,
  least(duration/3600,2)/2 AS duration_adj,
  current_metric.draw    AS current_draw,
  current_metric.steam   AS current_steam,
  current_metric.hpdt    AS current_hpdt,
  current_metric.moisture AS current_moisture,
  current_metric.freeness AS current_freeness,
  current_metric.speed    AS current_speed,
  coalesce(current_test.current_quality_test_1, current_metric.predictive_quality_test_1) AS quality_test_1,
  coalesce(current_test.current_quality_test_2, current_metric.predictive_quality_test_2) AS quality_test_2
JOIN current_metric USING ()
JOIN current_test   USING ()
WHERE time FROM date_trunc(now()-7M, 'D') TO date_trunc(now(), 'D')
  AND line.id    = '<LINE_ID_1>'
  AND product.id = '<PRODUCT_ID_1>'
  AND interval.state.type = 'Uptime'
GROUP BY interval.stable_period AS id
HAVING duration >= 3600
APPLY honk2science candidate_score(
  user_interest_cols = ['draw','steam','hpdt'],
  user_interest_metric_groups = ['<USER_INTEREST_MG_UUID_1>','<USER_INTEREST_MG_UUID_2>','<USER_INTEREST_MG_UUID_3>'],
  quality_test_identifiers = ['quality_test_1','quality_test_2'],
  model_identifiers = ['<MODEL_IDENTIFIER_1>','<MODEL_IDENTIFIER_2>'],
  constraint_measure_cols = ['moisture','freeness','speed'],
  constraint_measure_scaling_factors = [100,200,200],
  candidate_duration_col = 'duration_adj',
  candidate_score_col = 'score',
  remove_cols = ['duration','duration_adj','current_draw','current_steam']
)
SETTINGS backend=v1;

Returns OqlData with an added candidate_score_col and with any remove_cols dropped.

Raises GRPCError – On invalid inputs or if no production model is found for a given model_identifier.

🔗 See also recommend • infer_from_model

`• compute_recommendation_adherence`

Compute adherence of the active recommendation (per bucket) against actual metric-group values, returning per-metric scores and a rolled-up recommendation score.

Arguments

Name	Type	Description
`input_data`	OqlData	Must include recommendations JSON and metric-group values.
`recommendations_col`	str	Column with JSON object of recommendations.
`metricgroup_cols`	list[str]	Actual metric columns to compare.
`metricgroup_uuids`	list[str]	UUIDs for each metric column (1:1 with `metricgroup_cols`).
`metricgroup_difference_functions`	list[str]	Difference fn per metric (e.g., `"absolute"`).
`metricgroup_lower_significant_thresholds`	list[float]	Lower “0-score” thresholds.
`metricgroup_lower_moderate_thresholds`	list[float]	Lower “0.5-score” thresholds.
`metricgroup_upper_significant_thresholds`	list[float]	Upper “0-score” thresholds.
`metricgroup_upper_moderate_thresholds`	list[float]	Upper “0.5-score” thresholds.
`bucket_col`	str	Timestamp bucket column.
`active_rec_col`	str	Active recommendation ID column.
`show_intermediate_calculations`	int	1 to include intermediate columns (default 0).

Example

AGGREGATE
  avg(metric_group('<MG_UUID_1>')) AS mg_1,
  avg(metric_group('<MG_UUID_2>')) AS mg_2,
  avg(metric_group('<MG_UUID_3>')) AS mg_3,
  JSON_QUERY(interval.process_ai_active_recommendation.value, '$.recommendation.process_metrics[*]') AS recs
WHERE TIME FROM '<START_TS_1>' TO '<END_TS_1>'
  AND line.id = '<LINE_ID_1>'
GROUP BY interval.process_ai_active_recommendation AS active_rec, bucket 1m AS bucket_start
APPLY honk2science compute_recommendation_adherence(
  recommendations_col = 'recs',
  metricgroup_cols = ['mg_1','mg_2','mg_3'],
  metricgroup_uuids = ['<MG_UUID_1>','<MG_UUID_2>','<MG_UUID_3>'],
  metricgroup_difference_functions = ['absolute','absolute','absolute'],
  metricgroup_lower_significant_thresholds = [5.2,5.4,7.6],
  metricgroup_lower_moderate_thresholds   = [4.2,3.4,2.6],
  metricgroup_upper_significant_thresholds = [6.2,6.4,8.6],
  metricgroup_upper_moderate_thresholds   = [7.2,8.4,9.6],
  bucket_col = 'bucket_start',
  active_rec_col = 'active_rec'
);

Returns Per active recommendation: active_rec, {metric}_start/end/final, and rolled-up final score over the time window.

Raises GRPCError – Schema mismatch or unequal list lengths for thresholds/functions. GRPCError – Negative thresholds.

🔗 See also recommend • recommendation_achievement

`• recommendation_achievement`

Compute a per-period “achievement score” for a recommendation, based on distance of realized conditions vs. recommended.

Arguments

Name	Type	Description
`input_data`	OqlData	Time-bucketed series with relevant process metrics.
`distance`	str	Distance metric (default `"euclidean"`).

Example

AGGREGATE
  avg(metric_group('<PROC_MG_UUID_1>')) AS '<PROC_SP_NAME_1>',
  avg(metric_group('<PROC_MG_UUID_2>')) AS '<PROC_PV_NAME_1>',
  avg(metric_group('<PROC_MG_UUID_3>')) AS '<PROC_PV_NAME_2>'
WHERE time FROM now()-14D TO now()
  AND interval.ppo_run.value NOT IN ('')
  AND interval.state.category.name IN ('Uptime')
  AND line.id = '<LINE_ID_1>'
GROUP BY bucket 1m AS Bucket, line.name AS Line, interval.ppo_run.value AS ppo_value, interval.ppo_run.start AS ppo_start
SETTINGS backend=v1, timezone='UTC'
APPLY honk2science recommendation_achievement(distance='euclidean');

Returns OqlData with [Bucket, ppo_start, recommendation_start, achievement_score].

Raises GRPCError – On invalid input data.

🔗 See also compute_recommendation_adherence • compute_distance

`• infer_from_model`

Run inference using a production model with metric-group inputs (missing values are imputed with 0.0).

Arguments

Name	Type	Description
`input_data`	OqlData	Input rows (features in `input_cols`).
`model_identifier`	str	Production model ID.
`input_cols`	list[str]	Column aliases passed to the model (ordered).
`input_metric_group_uuid`	list[str]	UUIDs matching `input_cols` (ordered).
`inference_col`	str \| None	Output prediction column name.
`keep_cols`	list[str] \| None	Columns to retain in the output.

Example

AGGREGATE
  avg(metric_group('<MG_UUID_1>')) AS m1,
  avg(metric_group('<MG_UUID_2>')) AS m2,
  avg(metric_group('<MG_UUID_3>')) AS m3
WHERE time FROM now()-10m TO now()
  AND line.id = '<LINE_ID_1>'
GROUP BY bucket 5m
APPLY honk2science infer_from_model(
  model_identifier = '<MODEL_IDENTIFIER_1>',
  input_cols = ['m1','m2','m3'],
  input_metric_group_uuid = ['<MG_UUID_1>','<MG_UUID_2>','<MG_UUID_3>'],
  inference_col = 'predicted_output',
  keep_cols = ['bucket','m1','m2','m3']
)
SETTINGS backend=v1;

Returns OqlData with keep_cols and inference_col.

Raises GRPCError – Invalid input data or no production model for model_identifier.

🔗 See also infer_from_oql • compute_regression_slos

`• infer_from_oql`

Model inference from arbitrary OQL-shaped inputs (may include offline quality, custom features, etc.). Missing values imputed with 0.0. Optionally returns class probabilities.

Arguments

Name	Type	Description
`input_data`	OqlData	Input rows (features in `input_cols`).
`model_identifier`	str	Production model ID.
`input_cols`	list[str]	Feature columns passed to the model.
`inference_col`	str \| None	Output prediction column.
`keep_cols`	list[str] \| None	Columns to include in output.
`predict_proba`	str	`"True"` (case-insensitive) to request probabilities for classification; otherwise False.

Example

AGGREGATE
  avg(metric_group('<MG_UUID_1>')) AS m1,
  lastIf(offline_quality.results.avgd_val, offline_quality.results.tag_name = '<QUALITY_TEST_NAME_1>') AS q1
WHERE time FROM now()-10m TO now()
  AND line.id = '<LINE_ID_1>'
GROUP BY bucket 5m
APPLY honk2science infer_from_oql(
  model_identifier = '<MODEL_IDENTIFIER_1>',
  input_cols = ['m1','q1'],
  inference_col = 'predicted',
  keep_cols = ['bucket','m1'],
  predict_proba = 'True'
)
SETTINGS backend=v1;

Returns OqlData with keep_cols and inference_col (and probabilities, if requested).

Raises GRPCError – Invalid input data or no production model for model_identifier.

🔗 See also infer_from_model

`• compute_regression_slos`

Compute regression SLOs per group and cross-group: Correlation, R², MAPE, MSE, Nsamples; also provides a group-normalized row.

Arguments

Name	Type	Description
`input_data`	OqlData	Rows that contain target & prediction (optionally grouped).
`target_col`	str	Column name for the target.
`prediction_col`	str	Column name for the prediction.
`group_by_col`	str	Group key (e.g., product name). Use empty string for global only.

Example

AGGREGATE
  avgIf(offline_quality.results.result, offline_quality.results.testid = '<QUALITY_TEST_NAME_1>') AS target_col,
  avg(metric_group('<MG_UUID_PRED>')) AS prediction_col,
  product.name AS product
WHERE line.id = '<LINE_ID_1>'
  AND time FROM now()-30D TO now()
GROUP BY bucket 5m
ORDER BY bucket DESC
HAVING target_col > 0
APPLY honk2science compute_regression_slos(
  target_col      = 'target_col',
  prediction_col  = 'prediction_col',
  group_by_col    = 'product'
);

Returns OqlData shaped like:

group	Correlation	R2	MAPE	MSE	Nsamples
Product 1	…	…	…	…	…
Product 2	…	…	…	…	…
Product Normalized	…	…	…	…	…
Cross Product	…	…	…	…	…

Raises GRPCError – Invalid input.

🔗 See also infer_from_model • infer_from_oql

`• linear_polyfit`

Fit a simple linear regression (degree 1) and return either the slope or a status vs. thresholds.

Arguments

Name	Type	Description
`input_data`	OqlData \| DataFrame	Two or more points required.
`warning_threshold`	float \| None	Optional warning threshold on slope.
`danger_threshold`	float \| None	Optional danger threshold on slope.

Returns Tuple (column_name, value) where column_name is "slope" or "status".

Raises ValueError – Fewer than 2 points.

🔗 See also compute_regression_slos

`• threshold_segmentation`

Segment a timeline using thresholds on selected metric groups, optionally splitting on interval changes (e.g., product), and with optional custom filtering of first/last batch/run.

Arguments

Name	Type	Description
`input_data`	OqlData	Bucketed rows with metrics & interval context.
`interval_cols`	list[str] \| None	Interval columns that define change points.
`threshold_metric_groups`	list[str]	Metric-group UUIDs for thresholding.
`threshold_file_identifier`	list[str]	`[factory_id, line_id]` (order matters).
`threshold_percentile`	str	Percentile label (e.g., `"p80"`).
`min_segment_duration_s`	int	Minimum segment length (seconds).
`bucket_size_s`	int	Bucket size (seconds).
`custom_filtering`	str \| None	e.g., `"filter_first_last_batch"`.
`batch_time_cols`	list[str] \| None	`[batch_start, batch_end]` when filtering batches.
`run_time_cols`	list[str] \| None	`[run_start, run_end]` when filtering runs.

Example See example block in your inputs; typical pattern:

AGGREGATE
  interval.state.event.start AS start_uptime,
  interval.state.event.end   AS end_uptime,
  product.id AS product_id,
  avg(metric_group('<THRESHOLD_MG_UUID_1>')) AS '<THRESHOLD_MG_UUID_1>',
  avg(metric_group('<THRESHOLD_MG_UUID_2>')) AS '<THRESHOLD_MG_UUID_2>'
WHERE TIME FROM '<START_TS_2>' TO '<END_TS_2>'
  AND interval.state.type = 'Uptime'
  AND line.id = '<LINE_ID_1>'
GROUP BY bucket 5m AS start_bucket, interval.state.event AS event
APPLY honk2science threshold_segmentation(
  interval_cols = ['product_id'],
  threshold_metric_groups = ['<THRESHOLD_MG_UUID_1>','<THRESHOLD_MG_UUID_2>'],
  threshold_file_identifier = ['<FACTORY_ID_1>','<LINE_ID_1>'],
  threshold_percentile = 'p80',
  min_segment_duration_s = 900,
  bucket_size_s = 300
)
SETTINGS backend=v1, timezone='UTC';

Returns OqlData with [line_id, start_timestamp, end_timestamp].

Raises InvalidInputError – On invalid inputs.

🔗 See also detect_ramp_up_from_downtime

`• detect_ramp_up_from_downtime`

Identify ramp-up periods after downtime using learned thresholds on one or more metrics.

Arguments

Name	Type	Description
`input_data`	OqlData	Bucketed uptime windows with metric averages.
`threshold_metric_groups`	list[str]	Metric-group UUIDs used for ramp-up detection.
`threshold_file_identifier`	list[str]	`[factory_id, line_id]` identity (order matters).
`threshold_percentile`	str	e.g., `"p80"`.
`min_segment_duration_s`	int	Minimum ramp-up length (default 600s).
`bucket_size_s`	int	Bucket size in seconds (default 60s).

Example

AGGREGATE
  interval.state.event.start AS start_uptime,
  interval.state.event.end   AS end_uptime,
  avg(metric_group('<METRIC_ID_1>')) AS '<METRIC_ID_1>'
WHERE TIME FROM '<START_TS_3>' TO '<END_TS_3>'
  AND line.id = '<LINE_ID_1>'
  AND interval.state.type = 'Uptime'
GROUP BY bucket 5m AS start_bucket, interval.state.event AS event
APPLY honk2science detect_ramp_up_from_downtime(
  threshold_metric_groups = ['<METRIC_ID_1>'],
  threshold_file_identifier = ['<FACTORY_ID_1>','<LINE_ID_1>'],
  threshold_percentile = 'p80',
  min_segment_duration_s = 600,
  bucket_size_s = 300
)
SETTINGS backend=v1, timezone='UTC';

Returns [line_id, ramp_up_start_timestamp, ramp_up_end_timestamp].

Raises InvalidInputError – On invalid inputs.

🔗 See also threshold_segmentation

`• dummy_cluster`

Attach a simple cluster index to each row (placeholder clustering utility).

Arguments

Name	Type	Description
`input_data`	OqlData	Source rows.
`remove_cols`	list[str]	Columns to drop after clustering.
`add_cols`	list[str]	New column names to add.
`features`	list[str]	Feature columns used by the clustering stub.
`n_clusters`	int	Number of clusters.

Returns OqlData with a cluster_index (default name) plus any add_cols.

🔗 See also segment_repeatability

`• segment_repeatability`

Compute repeatability between temporal segments from both process and outcome distances; aggregate via the median of num_neighbors.

Arguments

Name	Type	Description
`input_data`	OqlData	Segment features & outcomes (wide table).
`id_cols`	list[str]	Segment identifier columns (e.g., `id`, `start`).
`process_cols`	list[str]	Process metric columns.
`outcome_cols`	list[str]	Outcome (quality/perf) columns.
`distance`	str	Distance function (e.g., `"mape"`, `"z-norm-euclidean"`).
`num_neighbors`	int	k for median across nearest neighbors.

Example

WITH stable (
  id,
  tensile_actual,
  tensile_pred,
  speed,
  start,
  end,
  basis_weight,
  tensile_lsl,
  tensile_usl,
  tensile_target,
  basis_weight_lsl,
  basis_weight_usl,
  basis_weight_target,
  v0,
  v1,
  v58
) AS (
  AGGREGATE
    avgIf(offline_quality.results.avgd_val, offline_quality.results.tag_name = '<QUALITY_TEST_NAME_2>') AS tensile_actual,
    avg(metric_group('<PREDICTION_MG_UUID_1>')) AS tensile_pred,
    avg(metric_group('<PERFORMANCE_MG_UUID_1>')) AS speed,
    interval.stable_period.start AS start,
    interval.stable_period.end   AS end,
    avgIf(offline_quality.results.avgd_val, offline_quality.results.tag_name = '<QUALITY_TEST_NAME_3>') AS basis_weight,
    avgIf(offline_quality.results.reject_low,  offline_quality.results.tag_name = '<QUALITY_TEST_NAME_2>') AS tensile_lsl,
    avgIf(offline_quality.results.reject_high, offline_quality.results.tag_name = '<QUALITY_TEST_NAME_2>') AS tensile_usl,
    avgIf(offline_quality.results.target,      offline_quality.results.tag_name = '<QUALITY_TEST_NAME_2>') AS tensile_target,
    avgIf(offline_quality.results.reject_low,  offline_quality.results.tag_name = '<QUALITY_TEST_NAME_3>') AS basis_weight_lsl,
    avgIf(offline_quality.results.reject_high, offline_quality.results.tag_name = '<QUALITY_TEST_NAME_3>') AS basis_weight_usl,
    avgIf(offline_quality.results.target,      offline_quality.results.tag_name = '<QUALITY_TEST_NAME_3>') AS basis_weight_target,
    avg(metric_group('<PROC_MG_UUID_4>')) AS headbox_wire_speed,
    avg(metric_group('<PROC_MG_UUID_5>')) AS headbox_jet_velocity,
    avg(metric_group('<PROC_MG_UUID_6>')) AS steam_flow_175,
    avg(metric_group('<PROC_MG_UUID_7>')) AS wetend,
    avg(metric_group('<PROC_MG_UUID_8>')) AS dryer_section_1_press,
    avg(metric_group('<PROC_MG_UUID_9>')) AS dryer_section_2_press,
    avg(metric_group('<PROC_MG_UUID_10>')) AS vent_stream_flow,
    avg(metric_group('<PROC_MG_UUID_11>')) AS headbox_slice_opening,
    avg(metric_group('<PROC_MG_UUID_12>')) AS v0,
    avg(metric_group('<PROC_MG_UUID_13>')) AS v1,
    avg(metric_group('<PROC_MG_UUID_14>')) AS v58
  WHERE TIME FROM date_trunc(now()-6M, 'D') TO date_trunc(now(), 'D')
    AND line.id = '<LINE_ID_1>'
    AND product.id = '<PRODUCT_ID_1>'
    AND interval.stable_period.value NOT IN ('')
  GROUP BY interval.stable_period
  SETTINGS backend=v1
)
AGGREGATE
  stable.id,
  stable.v0,
  stable.v1,
  stable.v58,
  stable.tensile_pred AS 'Tensile MD',
  stable.speed AS 'Machine Speed (ft/min)',
  stable.start,
  stable.end,
  stable.basis_weight AS 'Basis Weight',
  stable.tensile_actual AS 'hide Tensile Actual',
  stable.tensile_lsl AS 'hide Tensile LSL',
  stable.tensile_usl AS 'hide Tensile USL',
  stable.tensile_target AS 'hide Tensile Target',
  stable.basis_weight_lsl AS 'hide Basis Weight LSL',
  stable.basis_weight_usl AS 'hide Basis Weight USL',
  stable.basis_weight_target AS 'hide Basis Weight Target'
JOIN stable USING ()
WHERE TIME FROM date_trunc(now()-6M, 'D') TO date_trunc(now(), 'D')
  AND line.id = '<LINE_ID_1>'
  AND product.id = '<PRODUCT_ID_1>'
HAVING 'Tensile MD' > 'hide Tensile LSL'
  AND 'Basis Weight' > 'hide Basis Weight LSL' AND 'Basis Weight' < 'hide Basis Weight USL'
  AND 'Machine Speed (ft/min)' > 0
APPLY segment_repeatability(
  id_cols = ['stable.start'],
  process_cols = ['stable.v0','stable.v1','stable.v58'],
  outcome_cols = ['Tensile MD'],
  distance = 'z-norm-euclidean',
  num_neighbors = 3
)
SETTINGS backend=v1;

Returns OqlData with segment identifiers and: - median_process_distance - median_outcome_distance - median_score (repeatability score)

Raises GRPCError – Invalid input.

🔗 See also compute_distance • dummy_cluster

`• discover_motifs`

Find recurrent time-series motifs via matrix profile (experimental).

Arguments

Name	Type	Description
`input_data`	bytes	Raw series payload.
`subsequence_size`	int	Window length for motif search.

Returns Tuple (OqlData, dict) with motif results and metadata.

🔗 See also mpsearch

`• mpsearch`

Run a matrix-profile-based search between search_start_time and search_end_time (experimental).

Arguments

Name	Type	Description
`input_data`	bytes	Raw series payload (aggregated time series bytes).
`search_start_time`	Timestamp	Start of search window.
`search_end_time`	Timestamp	End of search window.

Examples

Example 1

AGGREGATE
  avg(metric('<METRIC_UUID_1>'))
WHERE (TIME FROM '<PATTERN_START_TS>' TO '<PATTERN_END_TS>' OR TIME FROM now()-1M TO now())
  AND line.id = '<LINE_ID_1>'
GROUP BY bucket 10s
APPLY honk2science mp_search(start_time = '<PATTERN_START_TS>', end_time = '<PATTERN_END_TS>');
-- org id <ORG_ID_1>

Sample Output

{"minimum_point": "<HIT_TS_1>", "mp_distance_profile": 4.369478225708008}
{"minimum_point": "<HIT_TS_2>", "mp_distance_profile": 9.4520902633667}
{"minimum_point": "<HIT_TS_3>", "mp_distance_profile": 7.951839447021484}
{"minimum_point": "<HIT_TS_4>", "mp_distance_profile": 4.002388954162598}

Returns Tuple (OqlData, dict) with search hits and metadata (e.g., distance profiles).

🔗 See also discover_motifs

`• compute_distance`

Compute distance from each row to a reference vector; optionally add a distance column, sort rows, and drop columns.

Arguments

Name	Type	Description
`input_data`	OqlData	Input rows.
`reference_vector_col_names`	list[str]	Feature columns in order.
`reference_vector_col_values`	list[float]	Reference values in the same order.
`remove_cols`	list[str] \| None	Columns to drop from result.
`add_distance_col`	str \| None	Name for distance column.
`sort_by`	str \| None	`"ascending"` (default) or `"descending"`.
`distance`	str	Distance metric (default `"euclidian"`).

Example

AGGREGATE
  p50(metric_group('<MG_UUID_1>')) AS metric_group1,
  p50(metric_group('<MG_UUID_2>')) AS metric_group2,
  interval.state.event.end AS end_uptime
WHERE TIME FROM date_trunc(now()-90D, 'D') TO date_trunc(now(), 'D')
  AND line.id = '<LINE_ID_1>'
  AND interval.state.type = 'Uptime'
GROUP BY bucket 5m, interval.custom_interval.value AS custom_interval
APPLY honk2science compute_distance(
  reference_vector_col_names = ['metric_group1','metric_group2'],
  reference_vector_col_values = [<METRIC_GROUP1_REF>, <METRIC_GROUP2_REF>],
  remove_cols = ['metric_group1'],
  add_distance_col = 'distance',
  sort_by = 'ascending',
  distance = 'euclidian'
)
SETTINGS backend=v1, timezone='UTC';

Returns OqlData with (optionally) add_distance_col and after applying remove_cols and sort_by.

Raises GRPCError – Invalid input.

🔗 See also segment_repeatability • recommendation_achievement

`• final_aggregate`

Perform “final pass” aggregations over an OQL result using JSON specs (sum/avg, weighted aggregations, transforms, counter rebasing, and passthrough columns).

Arguments

Name	Type	Description
`input_data`	OqlData	Source rows after your main aggregation.
`group_by_cols`	list[str]	Columns to group the final aggregation by.
`aggregations`	list[str]	JSON strings describing each output column.

Examples Aggregation:

APPLY honk2science final_aggregate(
  group_by_cols = ['product'],
  aggregations = [
    '{"output_col":"x","input_col":"c1","weight_by":"c0","function":"sum","precision":2}',
    '{"output_col":"y","input_col":"c2","weight_by":"","function":"avg"}'
  ]
);

Transform with passthrough

APPLY honk2science final_aggregate(
  aggregations = [
    '{"output_col":"rebased_tls","input_col":"TLS","function":"rebase_counter_by_min","passthrough_cols":["sampled_on"]}'
  ]
);

Returns OqlData with only the requested outputs (plus any passthroughs).

Raises GRPCError – Invalid input.

🔗 See also compute_regression_slos

`• accumulate_cost_savings`

Accumulate Total Cost, Projected Total Initial Cost, and Total Cost Savings from a time-bucketed cost-rate series. If input_data is null, returns nulls.

Arguments

Name	Type	Description
`input_data`	OqlData	Bucketed cost-rate time series.
`add_cols`	list[str]	Names of new columns (order matters). Defaults: `["Total Cost","Projected Total Initial Cost","Total Cost Savings"]`.
`cost_rate_col`	str	Name of cost-rate column.
`initial_cost_rate`	float	Baseline cost rate at session start (0 ⇒ use first bucket).
`cost_rate_resolution_in_min`	int	Minutes per bucket (typically 1).
`calculation`	str	`"absolute"` or `"percentage"`.

Example

AGGREGATE
  avg(metric_group('<COST_RATE_MG_UUID>')) AS "Cost Rate"
FROM TIME date_trunc('<SESSION_START_TS>','m') TO date_trunc(NOW(),'m')
WHERE line.id = '<LINE_ID_1>'
GROUP BY bucket 1 minute
HAVING "Cost Rate" > 0
APPLY honk2science accumulate_cost_savings(
  cost_rate_col = 'Cost Rate',
  initial_cost_rate = 5,
  cost_rate_resolution_in_min = 1,
  calculation = 'absolute',
  add_cols = ['Total Cost','Projected Total Initial Cost','Total Cost Savings']
)
SETTINGS backend=clickhouse;

Returns OqlData with the three accumulation columns.

Raises GRPCError – cost_rate_col missing. GRPCError – add_cols not exactly 3 names.

🔗 See also final_aggregate

`• recommend_performance_recipe`

For each combination (e.g., product×line), surface the percentile most performant stable segment and return its “recipe” metrics, plus average performance, opportunity %, and hours vs. target.

Arguments

Name	Type	Description
`input_data`	OqlData	Stable-segment rows with recipe & performance columns.
`remove_cols`	list[str]	Columns to drop before returning.
`add_cols`	list[str]	New columns (order matters). Defaults: `["Avg Performance","Opportunity Duration","% Opportunity"]`.
`combination`	list[str]	Grouping keys (one output row per combination).
`performance_col`	str	Performance column to rank/select.
`duration_col`	str	Segment duration column.
`target_col`	str	Performance target column.
`percentile`	float	Percentile to surface (default 0.95).

Example

AGGREGATE
  avg(<RECIPE_METRIC_1>) AS RECIPE_METRIC_1,
  avg(<RECIPE_METRIC_2>) AS RECIPE_METRIC_2,
  avg(<RECIPE_METRIC_3>) AS RECIPE_METRIC_3,
  avg(<RECIPE_METRIC_4>) AS RECIPE_METRIC_4,
  avg(metric_group('<PERFORMANCE_MG_UUID>')) AS Recipe_Performance,
  min(interval.run.target('<TARGET_MG_UUID>').lsl) AS Target
WHERE TIME FROM now()-60D TO now()
  AND line.id = '<LINE_ID_1>'
GROUP BY
  interval.ppo_stable AS "Segment Start",
  interval.run.product.id AS "Product",
  line.id AS "Line"
APPLY honk2science recommend_performance_recipe(
  combination = ['Product','Line'],
  performance_col = 'Recipe_Performance',
  target_col = 'Target',
  percentile = 0.95,
  remove_cols = ['Segment Start'],
  add_cols = ['Performance Avg','% Opportunity','Opportunity Hours']
)
SETTINGS backend=v1;

Returns OqlData with [combination keys, recipe metrics, performance, target, Avg Performance, % Opportunity, Opportunity Hours].

Raises GRPCError – combination/remove_cols columns missing. GRPCError – add_cols not exactly 3 names.

🔗 See also final_aggregate • recommend