#ICPE2026 workshop presentation "Detecting past and future change points in performance data for education and practice"

 Excited to share that we will be presenting at the ICPE 2026 Workshop (WEPPE) on May 4th in Florence, Italy.

Our talk, "Detecting past and future change points in performance data for education and practice", focuses on practical approaches to identifying performance anomalies and change points—both retrospectively and predictively.

The presentation will also cover the core principles behind the Perfomalist app (www.perfomalist.com), demonstrating how these methods can be applied in real-world IT capacity management and performance analysis scenarios.
Looking forward to engaging discussions with researchers and practitioners in performance engineering!

ABSTRACT:

Typical tasks in performance engineering education and practice include anomaly detection, outlier detection, change point detection, and trend analysis, with the goal of anticipating future threshold breaches and capacity risks. This paper presents Statistical Exception and Trend Detection (SETDS), a method for detecting both past and future change points in performance time series and discusses its use as both an analytical technique and a teaching tool. The paper provides an intuitive yet rigorous description of the SETDS method and demonstrates its practical application through Perfomalist, a free web-based tool that implements the approach. Perfomalist enables students and practitioners to visualize anomalies, trends, and change points using IT Control Charts and Exception Values, making advanced concepts in performance analysis more accessible. Perfomalist was also used as a baseline tool in a CMG.org Hackathon, where participants were tasked with identifying change points and anomalies in time-stamped performance data to reveal distinct phases and patterns. Finally, the method and tool have been successfully used in an online course on Performance Anomaly Detection offered through CMG.org, illustrating how real-world performance data and tooling can be effectively integrated into performance engineering education.

#Claude #AI has Built a Perfomalist #ChangePoint Client - README and validation

Below is readme file generated by Claude code for the task  "How #Claude #AI Built a Perfomalist #ChangePoint Client" from the previous post and quick graphical validation.

___________________________________ 

# Perfomalist API Change Point Detection

Programs to call the Perfomalist Control Points API and identify the 3 most significant change points in time-stamped data.

## Available Versions

- **Python**: `perfomalist_changepoints.py`
- **R**: `perfomalist_changepoints.R`

## API Overview

The Perfomalist API analyzes time series data for control points (change points) using statistical methods.

**Endpoint**: `POST https://api.perfomalist.com/api/controlpoints.py`

## Installation

### Python Requirements

```bash
pip install requests
```

### R Requirements

```r
install.packages(c("httr", "readr"))
```

## Usage

### Python

#### Quick Start (with sample data)

```bash
python perfomalist_changepoints.py
```

#### Using Your Own Data

1. **Modify the script** to load your CSV file:

```python
# In the main() function, replace:
data = create_sample_data()

# With:
data = load_data_from_csv('your_data.csv')
```

2. **CSV Format**: Your file should have these columns:
   - `Date` (e.g., "7/2/2011")
   - `Hour` (e.g., "0", "1", "2", ...)
   - `Value` (numeric)

3. **Adjust parameters** (optional):

```python
api_client = PerfomalistAPI(
    s_value=99,          # Statistical band (0-100)
    e_value=5,           # Exception threshold (%)
    baseline_length=7    # Baseline period (days)
)
```

### R

#### Quick Start (with sample data)

```bash
Rscript perfomalist_changepoints.R
```

#### Using Your Own Data

1. **Modify the script** to load your CSV file:

```r
# In the main() function, replace:
data <- create_sample_data()

# With:
data <- load_data_from_csv("your_data.csv")
```

2. **Adjust parameters** (optional):

```r
config <- list(
  s_value = 99,           # Statistical band (0-100)
  e_value = 5,            # Exception threshold (%)
  baseline_length = 7     # Baseline period (days)
)
```

## API Parameters

### sValue (Statistical Band)
- Range: 0-100
- 100 = UCL (Upper Control Limit) = MAX
- 0 = UCL = LCL = mean
- Default: 99
- Higher values create wider control bands

### eValue (Exception Value)
- Exception threshold as % of historical average
- Default: 5
- Determines sensitivity to deviations

### BaseLineLength
- Number of time periods to compare against
- Default: 7
- Acts as the baseline/comparison window

## Input Data Format

Your CSV should follow this structure:

```csv
Date,Hour,Value
7/2/2011,0,236274
7/2/2011,1,215359
7/2/2011,2,170011
7/2/2011,3,142371
...
```

## Output

The programs will:

1. **Display the raw API response** - so you can see all data returned
2. **Parse the response** - extract change points from the API output
3. **Rank by significance** - identify the most important change points
4. **Show top 3 change points** - with all relevant metrics

### Example Output

```
======================================================================
Perfomalist API - Change Point Detection
======================================================================

Using sample data...
Loaded 24 data points

Calling Perfomalist API...

API Response received
----------------------------------------------------------------------
[API response displayed here]
----------------------------------------------------------------------

Found 15 total data points in response

======================================================================
TOP 3 MOST SIGNIFICANT CHANGE POINTS:
======================================================================

#1 Change Point:
  Date: 7/2/2011
  Hour: 13
  Value: 285471
  Signal: 2.45
  Deviation: 3.2

#2 Change Point:
  Date: 7/2/2011
  Hour: 5
  Value: 126836
  Signal: -2.1
  Deviation: 2.8

#3 Change Point:
  Date: 7/2/2011
  Hour: 22
  Value: 255439
  Signal: 1.8
  Deviation: 2.3
```

## Programmatic Usage

### Python - As a Module

```python
from perfomalist_changepoints import PerfomalistAPI, load_data_from_csv

# Load your data
data = load_data_from_csv('mydata.csv')

# Initialize client
api = PerfomalistAPI(s_value=99, e_value=5, baseline_length=7)

# Prepare input
csv_input = api.prepare_csv_input(data)

# Call API
response = api.call_api(csv_input)

# Parse results
change_points = api.parse_response(response)

# Get top 3
top_3 = api.find_top_changepoints(change_points, top_n=3)

for cp in top_3:
    print(cp)
```

### R - Source as Library

```r
source("perfomalist_changepoints.R")

# Load your data
data <- load_data_from_csv("mydata.csv")

# Call API
response_text <- call_perfomalist_api(
  data,
  s_value = 99,
  e_value = 5,
  baseline_length = 7
)

# Parse and find top change points
response_df <- parse_response(response_text)
top_3 <- find_top_changepoints(response_df, top_n = 3)

print(top_3)
```

## Troubleshooting

### API Returns Error
- Check your internet connection
- Verify the API endpoint is accessible: www.perfomalist.com
- Ensure your data format matches the expected CSV structure

### No Change Points Found
- Try adjusting the `eValue` parameter (lower = more sensitive)
- Check if your data has enough variation
- Verify the `baseline_length` is appropriate for your data frequency

### Parsing Issues
- The script will display the raw API response for manual inspection
- Check that column names in the response match expected patterns
- You may need to customize the parsing logic based on the actual API response format

## API Response Format Note

The actual response format from the Perfomalist API may vary. The programs include flexible parsing logic to handle common formats, but you may need to inspect the raw response and adjust the parsing if the format differs from expectations.

## More Information

- Website: www.perfomalist.com
- Download sample data and documentation from the website

## License

These scripts are provided as-is for use with the Perfomalist API service.
 

_______________

Below is the quick graphical validation showing where the 3 change points were found:

 

How #Claude #AI Built a Perfomalist #ChangePoint Client

Below is a task I gave to Claude (https://claude.ai/). It took about 10 minutes to generate client applications in both Python and R.

Prompt:

Here is the task for you. There is a Perfomalist API described below. I need a program (in Python or R) that calls the API to identify the three most significant change points in date-stamped data.

API description: https://www.trutechdev.com/2021/11/the-change-points-detection-perfomalapi.html

Claude’s response is shown below.

__________________________________________________________________________

6:52 PM

I'll create a Python program to call the Perfomalist API and find the 3 most significant change points. Let me start by reading the relevant skills for this task.

Check available skills for API work and data handling

Now I'll create a Python program that calls the Perfomalist API to find change points:

Create Python script to call Perfomalist API and find top 3 change points

Now let me create an R version as well:

Create R script to call Perfomalist API and find top 3 change points

Now let me create a README with usage instructions:

Create README with usage instructions

Now let me create a sample data file you can use for testing:

Create sample CSV data file for testing

Perfect! Now let me move all the files to the outputs directory:

Ran a command, read a file

Perfect! I've created complete programs in both Python and R to call the Perfomalist API and find the 3 most significant change points. Here's what you get:

📦 Files Provided

1. perfomalist_changepoints.py - Full-featured Python implementation
2. perfomalist_changepoints.R - Complete R implementation
3. README.md - Comprehensive usage guide
4. sample_data.csv - Test data to get started

🚀 Quick Start

Python:

bash

pip install requests

python perfomalist_changepoints.py

R:

r

install.packages(c("httr", "readr"))

Rscript perfomalist_changepoints.R

Both programs will run with sample data out-of-the-box!

✨ Key Features

• ✅ Configurable API parameters (sValue, eValue, BaseLineLength)
• ✅ Automatic CSV formatting with parameters
• ✅ Flexible response parsing
• ✅ Intelligent change point ranking by significance
• ✅ Easy to use with your own CSV data
• ✅ Can be used as standalone scripts or imported as modules/libraries

To use your own data, just modify the script to load your CSV file instead of the sample data. The README has detailed instructions!

README file is published in the next post: https://www.trutechdev.com/2026/02/claude-ai-has-built-perfomalist.html

Link to CMG’25 Hackathon guidelines

CMG’25 Hackathon guidelines 

Below is one example from the document of using Performalist.com API:

The result (hacks) could be sent to trutechdev@gmail.com 

CMG’25 Hackathon guidelines 

ANNONCE (to publish before conference)

The task is to find some change points and/or anomalies in the given time stamped data to see different phases/patterns .

Participants can use any tools or libraries/packages (in R, Python and so on) to detect change points and/or anomalies in the data.

Or they can use the free Change Points detection API (e.g. via Postman.com) described in 

Change Point Detection is implemented in the free web tool Perfomalist

Particularly the instruction on how to use that is described HERE.

To visualize the result any spreadsheet charting could be used or other means (e.g. python, R and so on). Examples are on the following picture: 

The time to work on the task - 3 hours. 

The result would be judged by CMG experts and the winner will get an award and time to make a short presentation. 

Vendors of similar tools are welcome to participate. 

If participants would like to use MATLAB for anomaly detection tools, we have a licensed version available here:

URL for event page: https://www.mathworks.com/licensecenter/classroom/4866200/

1. this will take you to a MathWorks account Sign-in page.

2. Create an account or use an existing account

3. Press "Access MATLAB Online" button followed by "Open MATLAB Online"

On site activities

Data to test: https://github.com/numenta/NAB/tree/master/data

Particularly the following csv files:

LINK TO THE FOLDER WITH DATA IS HERE

EXAMPLE 1

Simple case: art_daily_flatmiddle.csv

Tool is performalist.com Change Point detection API described HERE

1st step to change the format of the data (using EXCEL or Google sheet means):

From original:

imestamp,value

2014-04-01 00:00:00,-21.0483826823

2014-04-01 00:05:00,-20.2954768676

2014-04-01 00:10:00,-18.127229468299998

2014-04-01 00:15:00,-20.1716653997

TO performalist form:

date,time,value

2014-04-01,0:00:00,-21.04838268

2014-04-01,0:05:00,-20.29547687

2014-04-01,0:10:00,-18.12722947

2014-04-01,0:15:00,-20.1716654

Then call perfomalist API (e.g. using postman.com)

The result is one change point on 2014-04-11, which can be easily validated by building the spreadsheet chart (see below):

EXAMPLE 2

More difficult case: https://github.com/numenta/NAB/blob/master/data/realTraffic/TravelTime_451.csv

After repeating the above steps (see Example 1)  the result should show several change points:

To reduce the number of change points one can explicitly to provide as a 1st 3 lines in the data the following tuning parameters:

• sValue - Statistical band in %, where 100 is UCL=MAX, 0 is UCL=LCL=mean). (normality)

• eValue - Exception Value (EV) threshold in % of actual historical average. (insensitivity)

• BaseLineLength - The time period to compare current value against.

After adding there

 sValue, 99

eValue, 20

BaseLineLength , 7

The API returns  only 3 change points:

Putting the API output to EXCEL or Google sheet one can visualize the result by showing phases in the data between change points (see below):

Anomaly/change detection hackathon proposal

ANNOUNCEMENT 

Objective: The goal of this hackathon is to identify change points and/or anomalies within a provided time-stamped dataset. This analysis will reveal distinct phases and patterns within the data.

Tools and Methods:

• Participants are encouraged to utilize any tools, libraries, or packages (e.g., R, Python) for change point and anomaly detection.

• Alternatively, participants can use the free Change Point Detection API via Postman (postman.com), as detailed below.

Resources:

• Change Point Detection API: Implemented in the free web tool Perfomalist: https://www.trub.in/2020/08/cpd-change-points-detection-is-planed.html 

• API Instructions: How to use the Change Points Detection Perfomalist  API: https://www.trutechdev.com/2021/11/the-change-points-detection-perfomalapi.html 

Hackathon Details:

• Time Limit: 3 hours to complete the task.

• Judging: CMG experts will evaluate the results.

• Award: The winner will receive an award and the opportunity for a short presentation.

• Vendor Participation: Vendors of similar tools are welcome to participate.

Visualization:

• Results can be visualized using spreadsheet charting tools (Excel, Google Sheets) or other methods (e.g., Python, R).

• Examples of visualizations will be provided below:

• 

"Detecting Past and Future Change Points in Performance Data" - research paper preprint

Our research paper was accepted for ORAL PRESENTATION at ICTDsC 2024 in India.

We were not able to go there and plan to publish that later at another place. The abstract is below.

And the paper itself could be found as a preprint at our google drive HERE

Perfomalist team is presenting at www.CMGimpact.com international conference in Orlando.

PRODUCT: www.Perfomalist.com

www.CMGimpact.com

LinkedIn Post

ABSTRACT: The MASF/SETDS method of detecting changes and anomalies in performance data, its recent implementation and the way to use and interpret the results will be presented with real examples against MangoDB testing data.

 

Since 1995, a time when the CMG conference published a very influential paper about the MASF method of anomaly detection, this topic has been increasingly more popular in the area of Capacity and Performance management. The modification of the MASF – SETDS method – was introduced in the 2002 CMG best paper and then got implemented in several companies and applications. This included the CMG online class “Perfomalies (Performance Anomaly) Detection”. Most recently this method has been turned into the cloud based serverless API microservice which is available for free via “Perfomalist” web app. The Perfomalist change points detection API was used against MongoDB’s testing data and got acceptable results which were published in the SPEC.org 2022 conference. This paper included an additional post-processing algorithm (XGBoost) to eliminate false positives which is planned to be added to Perfomalist service.