DeepFlow Milestones

This document was translated by GPT-4

As of now (September 2023), DeepFlow has iterated to v6.3.9. The main supported features are:

  • Universal Map (AutoMetrics)
    • TCP network performance metrics based on cBPF
    • Application protocol parsing and performance analysis based on cBPF
    • Application protocol parsing and performance analysis based on eBPF tracepoint/kprobe
    • File read and write performance indicators based on eBPF
    • Golang application HTTP2, gRPC protocol parsing, and performance analysis based on eBPF uprobe
    • Golang/openssl HTTPS protocol parsing and performance analysis based on eBPF uprobe
    • Use Wasm, so plugins to enhance the parsing of standard protocols, and to achieve private protocol parsing
  • Distributed Tracing (AutoTracing)
    • Support for eBPF-based non-intrusive distributed tracing, supporting any application with same-thread, Golang applications with cross-coroutine call chains
    • Use of Wasm/so plugins to extract business serial numbers, implementing non-intrusive distributed tracing only dependent on cBPF
    • Support for tracing call chains before and after ALB by parsing X-Request-ID fields in requests
    • Support for tracking MySQL call chains by parsing MySQL Comment fields in requests
    • Support for tracking cross-thread call chains by parsing TraceID/SpanID fields in requests
    • Support for automatically tracing the network forwarding path, calculating the performance per hop
  • Continuous Profiling (AutoProfiling)
    • Support for eBPF non-intrusive collection of Profile data for C/C++/Golang/Rust/Java applications
    • Support for eBPF non-intrusive collection of OnCPU Profile
  • AutoTagging & SmartEncoding
    • Automatic injection of cloud resource tags, K8s service tags
    • Automatic injection of K8s Label/Annotation/Env custom labels
    • Support for injecting business tags in CMDB via API
    • Support for injecting process business labels exposed by the business release system via Plugin
    • High-performance SmartEncoding label storage query capabilities
  • Integration
    • Support for integrating metrics data from Prometheus and Telegraf
    • Support for integrating tracing data from OpenTelemetry and SkyWalking and associating it with eBPF tracing data to achieve full-stack distributed tracing
    • Support for integrating continuous performance profiling data from Pyroscope
    • Support for SQL/PromQL API, support for being a Grafana DataSource
    • Support for displaying distributed tracing data in Grafana Tempo
    • Support for exporting tracing data to OpenTelemetry Collector
    • Support for exporting tracing data to SkyWalking

There are many exciting future plans for DeepFlow:

  • Universal Map (AutoMetrics)
    • Enhanced protocol parsing ability based on TCP stream reassembly
    • Built-in parsing capability for more application protocols, such as Elasticsearch
    • Support for more types of protocol parsing plugins, such as Lua
    • Support for eBPF collection of more network and system performance indicators within the call lifecycle
  • Distributed Tracing (AutoTracing)
    • eBPF-based non-intrusive distributed tracing capability supports more complex scenarios such as cross-thread, Golang Channel
    • Real-time aggregation and calculation of Trace topology based on Span (while preserving the call chain relationship), to reduce the storage overhead of tracing data
  • Continuous Profiling (AutoProfiling)
    • Support for eBPF non-intrusive collection of more types of Profile data, such as OffCPU, Memory
    • Elimination of Profile data's dependency on application process symbol tables
    • Support for non-intrusive collection of Profile data for Node.js/Python/PHP/Ruby applications
  • AutoTagging & SmartEncoding
    • Automatic injection of business labels entered by the tenant on the cloud platform
  • Integration
    • Support for integration with Sentry RUM data source
    • Support for integration with Promtail/Loki log data source
    • Support for displaying log data in Grafana Loki GUI
    • Support for outputting eBPF metrics data as a Prometheus Exporter
    • Support for exporting tracing data to Kafka