How it works

Easy to start

Often developers avoid performance testing because it seems complicated: you need to write special tests, run them correctly, use dedicated servers for them. Yes, it's complicated. But in order to start performance testing it’s enough to have usual unit, functional or integration tests.

Perfvane detects anomalies in durations of your unit, functional or integration tests.

Definitions

Perfvane uses the term performance anomaly: a situation when duration of a test looks "strange". Several examples of anomalies:

  • Degradation. The test was fast before, and became slower.
  • Acceleration. The test was slow before, and became faster.
  • Multimodal distributions. The distribution has several modes.

Algorithm

Currently Perfvane detects three types of anomalies: Degradation, Acceleration and general Changepoint. Changepoint may be any other change: the variance is increased, or unimodal distribution became multimodal.

The service is based on the changepoint detection algorithm RqqPelt described in an open-source project Perfolizer.

Workflow

Retrospective analysis

Immediately after you add a GitHub project Perfvane starts loading early builds for 1 year and analyzes history of each test durations.

Please note that loading early builds may take some time.

If performance anomalies is found in the past they will be appear on Anomalies page..

Every commit analysis

After retrospective analysis Perfvane starts analyzing every new build:

  1. You push code to GitHub.
  2. Your CI Provider runs tests.
  3. Perfvane loads build test report.
  4. Perfvane analyze history of each tests' durations with RqqPelt algorithm.