RVV benchmark Tenstorrent Ascalon X™

What's going on with the comparison instructions?

If you study the instruction throughput measurements below, you may notice something confusing: The LMUL>1 vector-scalar/immediate (vmseq.vx/vmseq.vi) instructions are two times faster than the vector-vector comparison instructions (e.g. vmseq.vv).
I initially suspected a measuring artifact, but on further contemplation, this looks like a very smart uarch optimization.

Ascalon likely has three VLEN-wide vector register file read ports, and a GPR read port. This can be seen from the measurements below. Notice how vadd.vv performs the same with vl=VLMAX ta/ma (would work with 2 vreg reads) as with vl=1 tu/mu (needs 3 vreg reads).

Now consider the registers file accesses from the vmseq.vv instruction. It reads two LMUL wide vector registers and writes a single mask register. The vmseq.vx/vi instructions read one LMUL wide vector register, one GPR, and write a single mask register. So the vmseq.vx/vi instructions only utilize one of the three available read ports. But for LMUL>1, why not use both of the available vreg read ports and double the ALU width for the comparison instructions? Widening the ALU is a lot cheaper compared to extra ports or execution units, the former scales linearly, the latter quadratically. This is a clever optimization I suspect Ascalon does, which results in 2x faster LMUL>1 vector comparisons.

The trick doesn't work for vmseq.vv because to process VLEN*2 elements at a time, you'd need to be able to read four VLEN-wide vector registers at once.
But why doesn't this trick work for other .vx/vi instructions like vadd.vx/vi? Because they need to write an LMUL-wide vector register. The comparison instructions write a mask result, and RVV stores all masks packed in the lower part of an LMUL=1 vector register, so even an LMUL=8 vmseq.v* instruction only needs to occupy the VLEN-wide write port once. (Although it may also write back partial results for improved ILP)

There are actually a few other instructions that write back <LMUL vector registers, namely vmadc.vxm/vim, vmadc.vxm/vim, the vector narrowing and reduction instructions. We can observe this optimization in those instructions as well, except for the reductions. (Maybe because reduction networks are more expensive as they don't scale linearly?)
This is very nice and, e.g. makes LMUL=n vnsrl.wv/wx/wi perform the same as LMUL=n/2 vsrl.vv.

If the assumption about the three vreg ports and the above optimization is correct, then a tail agnostic vnsrl.wv/wx/wi instruction fully utilizes the three VLEN-wide vreg ports while processing VLEN*2 elements at a time, one for the shift amount and two for the wide vector we are narrowing. But for tail undisturbed, we additionally need to read the destination, so the optimization doesn't work out, and indeed we can see that vnsrl.wv/wx/wi is 2x slower with tail undisturbed vs agnostic in the measurements below.

TLDR: Ascalon seems to use a clever uarch trick to make LMUL>1 vector-scalar/immediate comparisons and narrowing instructions twice as fast.

I'll start preferring .vx/.vi for LMUL>1 comparisons from now on, even if the register pressure permits preloading a vector register.