In the past I used the refined topological vertex and remember the confusion I encountered when trying to compute with it. I did not investigate it deeply at the time, but I noticed some inconsistencies in the paper that introduced the refined vertex (The Refined Topological Vertex). I recently returned to the formula with Claude and Mathematica. This traced part of my old confusion to a notation error in the original 2007 paper.

This is a nice showcase of how LLMs are able to help even with somewhat niche topics of physics and mathematics.

The Setup

The refined topological vertex is a combinatorial building block for computing partition functions of topological string theory on toric Calabi-Yau geometries. It was introduced by Iqbal, Kozcaz, and Vafa (IKV) in 2007 and depends on two parameters $(q, t)$ instead of the single coupling of the unrefined vertex.

The setup on my side was simple: a research wiki that the agent could read and write, Claude as the agent, and Mathematica for the algebra. I asked Claude to compute a couple of example geometries and document them as step-by-step derivations, checking the results against independent answers. Nobody had asked it to verify the paper. During those checks, Claude noticed a problem with the arm and leg convention in the formula, and I asked it to investigate whether the discrepancy came from our implementation or from the paper itself.

The Vertex in Brief

The vertex is a sum over Young diagrams. For a box at position $(i,j)$ in a diagram $\nu$, two standard quantities appear everywhere in this subject:

the arm length $a_\nu(i,j) = \nu_i - j$, the number of boxes to the right,
the leg length $\ell_\nu(i,j) = \nu_j^t - i$, the number of boxes below.

The vertex depends on a function $\tilde{Z}_\nu(t,q)$, defined in equation (25) of the paper (the same definition recurs at equations (12) and (85)):

\[\tilde{Z}_\nu(t,q) = \prod_{(i,j) \in \nu} \frac{1}{1 - t^{a(i,j)+1}\, q^{\ell(i,j)}}\]

together with inline definitions $a(i,j) = \nu_j^t - i$ and $\ell(i,j) = \nu_i - j$.

A closer look at these definitions shows the problem. The letter $a$ is defined as what is universally called the leg, and $\ell$ as the arm. The equation is self-consistent if you use its inline definitions. But anyone who sees the letters $a$ and $\ell$ and reaches for the standard arm and leg, which the paper itself draws in its Appendix A, implements the formula with the two exponents swapped.

An Independent Check

This is the issue. I will call it the face-value reading: the letters taken at face value, with $a_\nu$ and $\ell_\nu$ carrying their standard meanings. It gives

\[\tilde{Z}_\nu^{\text{face-value}}(t,q) = \prod_{s \in \nu} \frac{1}{1 - t^{a_\nu(s)+1}\, q^{\ell_\nu(s)}}\,.\]

This can be checked using the resolved conifold, for which the paper computes the partition function in two ways. The first uses a sum over Young diagrams built from $\tilde{Z}_\nu$ (equation (64)). The second uses Schur-function identities that do not depend on the arm and leg product and gives the closed form in equations (67)-(68):

\[Z(t,q,Q) = \prod_{i,j \geq 1} \left(1 - Q\, q^{i-\frac{1}{2}}\, t^{j-\frac{1}{2}}\right).\]

I can therefore use the second result as an independent check. Expanding both expressions in $Q$ and subtracting the closed form from the face-value result gives

\[Z^{\text{face-value}} - Z = -\,Q^2\, (q-t)^2\, \frac{q\,t\,(q+t)}{(1-q)^2(1+q)\,(1-t)^2(1+t)\,(1-qt)} + \mathcal{O}(Q^3)\,.\]

The terms at order $Q$ agree, but at order $Q^2$ the difference is not zero. Its factor of $(q-t)^2$ also shows that setting $q=t$, the usual unrefined check, cannot reveal the problem. The direct alternative reading is to swap the arm and leg lengths in the exponents:

\[\tilde{Z}_\nu^{\text{intended}}(t,q) = \prod_{s \in \nu} \frac{1}{1 - t^{\ell_\nu(s)+1}\, q^{a_\nu(s)}}\,.\]

With this definition, the coefficients agree analytically with the independent product formula.

This is not a random change made to force agreement. Equation (25) uses the letters $a$ and $\ell$ differently from their standard meaning in the paper’s own appendix. Swapping them is therefore the direct alternative reading of the formula.

Why It Is Easy to Miss

The factor $(q-t)^2$ explains why this mistake is so easy to miss:

One sanity check for anything refined is the unrefined limit $t = q$. There both readings collapse to the same expression, so every unrefined test passes with the wrong formula.
The error is not even linear in $(q-t)$, so first-order expansions around the unrefined limit pass too.
At degree one in $Q$ the two readings agree, because the single-box diagram is its own transpose, so the simplest refined check also passes.

There is a structural reason for this behavior. Transposing a Young diagram swaps its arms and legs, so the face-value reading is the intended function evaluated on the transposed diagram:

\[\tilde{Z}_\nu^{\text{face-value}}=\tilde{Z}_{\nu^t}.\]

Inside the partition sum, this introduces an additional factor

\[\left(\frac{t}{q}\right)^{\kappa(\nu)/2}, \qquad \kappa(\nu)=\lVert\nu\rVert^2-\lVert\nu^t\rVert^2.\]

This is a framing twist. At $t=q$ it equals one, which explains why the unrefined limit cannot distinguish the two readings.

The linear term also vanishes. Under transposition, $\kappa(\nu)$ changes sign, while the rest of the summand at $t=q$ remains unchanged. The contributions from $\nu$ and $\nu^t$ therefore cancel in pairs. The first possible difference is quadratic in $(q-t)$.

The same factor prevents the direct use of the Cauchy identity that produces the paper’s closed product. It depends on the shape of the diagram, not only on its degree $\lvert\nu\rvert$, so it cannot be absorbed into the parameter $Q$.

The Paper Against Itself

The conifold comparison shows which reading gives the correct partition function. The appendix makes the source of the mismatch visible directly. Figure 23 (page 59 of the arXiv version) defines $a$ and $\ell$ pictorially in the standard way, displays the general product $(1 - t^{a+1}q^{\ell})^{-1}$, and then displays a worked example,

\[\tilde{Z}_{(4,2,1)}(t,q) = \frac{1}{(1-t)^3\,(1-tq)\,(1-t^2q)\,(1-t^2q^2)\,(1-t^3q^3)}\,.\]

Working out the general formula as printed, with the figure’s own $a$ and $\ell$, gives instead

\[\tilde{Z}_{(4,2,1)}^{\text{face-value}}(t,q) = \frac{1}{(1-t)^3\,(1-t^2)\,(1-t^2q)\,(1-t^3q)\,(1-t^4q^2)}\,,\]

visibly different factors. The printed example agrees with the intended reading, $t^{\ell+1}q^{a}$ in standard names, and contradicts the general formula two lines above it.

I expected equation (149) directly below, which identifies $\tilde{Z}_\nu$ as a principal specialization of a Macdonald function, to settle the question the same way. It settles it the other way. As printed,

\[\tilde{Z}_\nu(t,q) = t^{-\lVert\nu\rVert^2/2}\, P_{\nu^t}(t^{-\rho};q,t), \qquad t^{-\rho} = (t^{1/2}, t^{3/2}, t^{5/2}, \dots),\]

equals the face-value product, not the intended one. The partition $\nu=(2)$ decides this free of any convention ambiguity: $\nu^t=(1,1)$ and $P_{(1,1)} = e_2$ for every $(q,t)$, so the right-hand side is $t^{-2}\, e_2(t^{1/2}, t^{3/2}, \dots) = 1/\big((1-t)(1-t^2)\big)$, with no $q$ in it at all, while the intended reading gives $1/\big((1-t)(1-tq)\big)$. The identification that matches the intended reading is

\[\tilde{Z}_\nu(t,q) = t^{-\lVert\nu^t\rVert^2/2}\, P_{\nu}(t^{-\rho};q,t)\,.\]

So the equations of the paper split into two internally consistent clusters: the worked example of Figure 23, the conifold computation, and the gauge-theory comparison use the intended function, while the Macdonald identifications (the first line of equation (25), equation (149), and equation (154)) are transposed in exactly the same way as the letters and hold only for the face-value product.

To be clear about what kind of error this is: the partition functions in the paper are correct. The product formula in equation (25), read strictly with its own inline definitions, produces the right function, and the authors evidently computed with the right object. What is wrong is the labeling, and with it the Macdonald identifications that repeat the same transposition. The wrong numbers appear only when a reader takes the letters, or the Macdonald formula, at face value. The computed results are internally consistent; the notation around them is not.

And this is where my confusion came from. Everyone implementing this formula faces the same silent choice between the letters and the standard convention, nothing in the usual checks reveals which branch you took, and different implementations end up disagreeing in ways that are miserable to track down.

Finding the error requires no individually difficult step. It requires keeping $q$ and $t$ distinct, expanding beyond degree one, comparing against an independent derivation, and checking the notation against the appendix. The value of the setup lies in carrying out that full chain. Claude reads the paper, translates the formulas into computations, and decides what to compare. Mathematica performs the symbolic algebra and checks whether the resulting expressions are equal.

Reproducing It

The symbolic checks in this post come from a single short Mathematica script. Its first part takes a partition, builds both readings of equation (25) from the arm and leg definitions, and prints the two factored products next to the Figure 23 comparison. Change the partition and it recomputes. Its second part derives the $(q-t)^2$ difference formula for the conifold at degree two, after checking that both readings agree at degree one. Its third part checks the $\nu=(2)$ Macdonald specialization. Running it verifies the main results above: the intended reading matches the product exactly, the face-value reading differs by a term with an explicit $(q-t)^2$ factor, the difference vanishes at $t = q$, the Figure 23 example agrees only with the intended reading, and equation (149) as printed agrees only with the face-value reading. The full algorithm for building partition functions from the vertex (gluing rules, framing, general geometries) is a story for another post.

References

The Refined Topological Vertex (Iqbal, Kozcaz, Vafa; equations (12), (25), (85), (149), (154), the conifold of Section 5.1, and Appendix A, Figure 23)
Young tableau (arm and leg lengths as standard combinatorial quantities)
Macdonald polynomial (conventions for $P_\lambda(x;q,t)$; $P_{(1^r)} = e_r$ for all parameters)