2025-02-11 Ordinary Meeting Notes

Feb 11, 2025 | RV-LFX Self-hosted Trace (1 hour)

Attendees: Beeman Strong

Notes

• Attendees: Beeman, Artem, Bo, Bruce, Daniel, Iain, Matthew, AndrewJ, Greg, Robert

• Slides, video

• Intros

  • Artem - former Intel, implemented PT support in Vtune for anomaly detection, virtualization support, etc.  Doing individual research now with RV and ARM ETM.

  • Andrew - Ventana, enabling extensions for Linux, virtualization as well.  Building framework for E-trace in Linux.

• Trace Address translation

  • Reviewed options considered for hypervisor-tracing-guest usage

    • Guest-physical addressing, requires mapping buffer into multiple tables

    • Use HV satp, adds complexity for nested HV tracing a guest, requires root HV to mirror pages

    • Use new trace-specific *tatp registers, described in following slides

  • Walked through illustrations of scenarios

    • Only need *tatp for cases where a HV is tracing a guest, otherwise can use translations from existing *atp regs

    • For HV tracing guest, htatp points to satp, so uses HV translation during guest execution

    • For nested HV tracing guest, htatp/hgtatp point to nested HV’s vsatp/hgatp, so uses nested HV’s translation during guest exec

  • *tatp always uses 2 stage translation?

    • Yes, though hgtatp may be in Bare mode (as in root HV tracing guest example)

    • And there can be cases where using 2-stage translation even when V=1

      • E.g., guest tracing itself, after trap to HS-mode

        • Either have to flush trace buffer before entering HS-mode, or let trace writes keep using vsatp/hgatp after the transition

      • Won’t have lots of trace after a trap

        • Trace generation stops but emitting buffered trace continues.  Likely 10s to 100s of bytes.

  • When to use *tatp register?

    • Want to be able to have multiple instances of this memory buffer, e.g. for trace and for sampling

    • Can’t assume that both want the same translation regime at the same time

      • E.g., could have hypervisor tracing guest and guest profiling itself

    • Could duplicate *tatp registers per buffer instance, but doesn’t scale well

    • Believe all buffers can share single pair of *tatp register, if each buffer can independently select whether *tatp or native regime is used

      • This wouldn’t support 2 HVs (e.g., nested and root) tracing/sampling the same guest though.  Think that’s acceptable.

  • How do we flush trace translations when switching trace translation regime?

    • Does SFENCE.VMA flush them?

    • AI Beeman to cover this in a future meeting and/or on email list

    • Think we can do the following:

      • Always flush on *tatp writes

      • If *tatp not in use, flush trace entries when flushing other entries using the same translation regime (e.g., on SFENCE.VMA)

Action items

• Feb 11, 2025 - Beeman Strong - cover trace translation flush scenarios