SINGULARITIES OF THETA DIVISORS AND THE GEOMETRY OF A_{5}

G. FARKAS, S. GRUSHEVSKY, R. SALVATI MANNI, AND A. VERRA

Abstract. We study the codimension two locus H in A_{g} consisting of principally polarized abelian varieties whose theta divisor has a singularity that is not an ordinary double point. We compute the class [H] ∈ CH^{2}(A_{g}) for every g. For g = 4, this turns out to be the locus of Jacobians with a vanishing theta-null. For g = 5, via the Prym map we show that H ⊂ A_{5} has two components, both unirational, which we describe completely. We then determine the slope of the effective cone of \overline{A}_{5} and show that the component \overline{N′_{0}} of the Andreotti-Mayer divisor has minimal slope.

Introduction

The theta divisor Θ of a generic principally polarized abelian variety (ppav) is smooth. The ppav (A, Θ) with a singular theta divisor form the Andreotti-Mayer divisor N_{0} in the moduli space A_{g}, see [AM67] and [Bea77]. The divisor N_{0} has two irreducible components, see [Mum83] and [Deb92], which are denoted θ_{null} and N′_{0}: here θ_{null} denotes the locus of ppav for which the theta divisor has a singularity at a two-torsion point, and N′_{0} is the closure of the locus of ppav for which the theta divisor has a singularity not at a two-torsion point. The theta divisor Θ of a generic ppav (A, Θ) ∈ θ_{null} has a unique singular point, which is a double point. Similarly, the theta divisor of a generic element of N′_{0} has two distinct double singular points x and −x. Using this fact, one can naturally assign multiplicities to both components of N_{0} such the following equality of cycles holds, see [Mum83],[Deb92]

(1)  N_{0} = θ_{null} + 2N′_{0}.

As it could be expected, generically for both components the double point is an ordinary double point (that is, the quadratic tangent cone to the theta divisor at such a point has maximal rank g — equivalently, the Hessian matrix of the theta function at such a point is non-degenerate). Motivated by a conjecture of H. Farkas [HF06], in [GSM08] two of the authors considered the locus in θ_{null} in genus 4 where the double point is not ordinary. In [GSM07] this study was extended to arbitrary g, considering the sublocus θ_{null}^{g−1} ⊂ θ_{null} parameterizing ppav (A, Θ) with a singularity at a two-torsion point, that is not an ordinary double point of Θ. In particular it has been proved that

(2)  θ_{null}^{g−1} ⊂ θ_{null} ∩ N′_{0}.

In fact the approach yielded a more precise statement: Let ϕ : X_{g} → A_{g} be the universal family of ppav over the orbifold A_{g} and S ⊂ X_{g} be the locus of singular points of theta divisors. Note that S can be viewed as a subscheme of X_{g} given by the vanishing of the theta functions and all its partial derivatives, see Section 1. Then S decomposes into three equidimensional components [Deb92]: S_{null}, projecting to θ_{null}, S′, projecting to N′_{0}, and S_{dec}, projecting (with (g − 2)-dimensional fibers) onto A_{1} × A_{g−1}. It is proved in [GSM07] that set-theoretically, θ_{null}^{g−1} is the image in A_{g} of the intersection S_{null} ∩ S′. An alternative proof of these results has been found by Smith and Varley [SV12a],[SV12b].

It is natural to investigate the non-ordinary double points on the other component N′_{0} of the Andreotti-Mayer divisor. Similarly to θ_{null}^{g−1}, we define N_{0}^{′g−1}, or, to simplify notation, H, to be the closure in N′_{0} of the locus of ppav whose theta divisor has a non-ordinary double point singularity. Note that H is the pushforward under ϕ of a subscheme H of X_{g} given by a Hessian condition on theta functions. In particular H can be viewed as a codimension 2 cycle (with multiplicities) on A_{g}. Since an explicit modular form defining N′_{0} and the singular point is not known, we consider the cycle

(3)  N_{0}^{g−1} := θ_{null}^{g−1} + 2N_{0}^{′g−1} = θ_{null}^{g−1} + 2H.

We first note that θ_{null}^{g−1} is a subset of H. Then, after recalling that the Andreotti-Mayer loci N_{i} are defined as consisting of ppav (A, Θ) ∈ A_{g} with dim Sing(Θ) ≥ i, we establish the set-theoretical inclusion N_{i} ⊂ H, for i ≥ 1. From this we deduce:

Proposition 0.1. For g ≥ 5 we have θ_{null}^{g−1} ⊊ H.

To further understand the situation, especially in low genus, we compute the class:

Theorem 0.2. The class of the cycle H inside A_{g} is equal to

[H] = (\frac{g!}{16}(g^{3} + 7g^{2} + 18g + 24) − (g + 4)2^{g−4}(2^{g} + 1)) λ_{1}^{2} ∈ CH^{2}(A_{g}).

As usual, λ_{1} := c_{1}(𝔼) denotes the first Chern class of the Hodge bundle and CH^{i} denotes the Q-vector space parameterizing algebraic cycles of codimension i with rational coefficients modulo rational equivalence. Comparing classes and considering the cycle-theoretic inclusion 3θ_{null}^{3} ⊂ H, we get the following result, see Section 4 for details:

Theorem 0.3. In genus 4 we have the set-theoretic equality θ_{null}^{3} = H.

We then turn to genus 5 with the aim of obtaining a geometric description of H ⊂ A_{5} via the dominant Prym map P : R_{6} → A_{5}. A key role in the study of the Prym map is played by its branch divisor, which in this case equals N′_{0} ⊂ A_{5}, and its ramification divisor Q ⊂ R_{6}. We introduce the antiramification divisor U ⊂ R_{6} defined cycle-theoretically by the equality

P^{∗}(N′_{0}) = 2Q + U.

Using the geometry of the Prym map, we describe both Q and U explicitly in terms of Prym-Brill-Noether theory. For a Prym curve (C, η) ∈ R_{g} and an integer r ≥ −1, we recall that V_{r}(C, η) denotes the Prym-Brill-Noether locus (see Section 5 for a precise definition). It is known [Wel85] that V_{r}(C, η) is a Lagrangian determinantal variety of expected dimension g − 1 − \binom{r+1}{2}. We denote π : R_{g} → M_{g} the forgetful map. Our result is the following:

Theorem 0.4. The ramification divisor Q of the Prym map P : R_{6} → A_{5} equals the Prym-Brill-Noether divisor in R_{6}, that is,

Q = {(C, η) ∈ R_{6} : V_{3}(C, η) ≠ 0} .

The antiramification divisor is the pull-back of the Gieseker-Petri divisor from M_{6}, that is, U = π^{∗}(GP_{6,4}^{1}). The divisor Q is irreducible and reduced.

As the referee pointed out to us, the irreducibility of Q (as well as that of U) also follows from Donagi’s results [Don92] on the monodromy of the Prym map P : R_{6} → A_{5}. Apart from the Brill-Noether characterization provided by Theorem 0.4, the divisor Q has yet a third (respectively a fourth!) geometric incarnation as the closure of the locus of points (C, η) ∈ R_{6} with a linear series L ∈ W_{6}^{2}(C), such that the sextic model φ_{L}(C) ⊂ P^{2} has a totally tangent conic, see Theorem 8.1 (respectively as the locus of section (C, η) ∈ R_{6} of Nikulin surfaces [FV11]). The rich geometry of Q enables us to (i) compute the classes of the closures \overline{Q} and \overline{U} inside the Deligne-Mumford compactification \overline{R}_{6}, then (ii) determine explicit codimension two cycles in R_{6} that dominate the irreducible components of H. In this way we find a complete geometric characterization of 5-dimensional ppav whose theta divisor has a non-ordinary double point. First we characterize θ_{null}^{4} as the image under P of a certain component of the intersection Q ∩ P^{∗}(θ_{null}):

Theorem 0.5. A ppav (A, Θ) ∈ A_{5} belongs to θ_{null}^{4} if an only if it is lies in the closure of the locus of Prym varieties P (C, η), where (C, η) ∈ R_{6} is a curve with two vanishing theta characteristics θ_{1} and θ_{2}, such that

η = θ_{1} ⊗ θ_{2}^{∨}.

Furthermore, θ_{null}^{4} is unirational and [θ_{null}^{4}] = 27 · 44 λ_{1}^{2} ∈ CH^{2}(A_{5}).

Denoting by Q_{5} ⊂ R_{6} the locus of Prym curves (C, η = θ_{1} ⊗θ_{2}^{∨}) as above, we prove that Q_{5} (and hence θ_{null}^{4} which is the closure of P (Q_{5}) in A_{5}) is unirational, by realizing its general element as a nodal curve

C ∈ |I_{R_{1}·R_{2}/P^{1}×P^{1}}^{2}(5, 5)| ,

where R_{1} ∈ |O_{P^{1}×P^{1}}(3, 1)| and R_{2} ∈ |O_{P^{1}×P^{1}}(1, 3)|, with the vanishing theta-nulls θ_{1} and θ_{2} being induced by the projections on the two factors.

Observing that [H] ≠ [θ_{null}^{4}] in CH^{2}(A_{5}), the locus H must have extra irreducible components corresponding to ppav with a non-ordinary singularity that occurs generically not at a two-torsion point. We denote by H_{1} ⊂ A_{5} the union of these components, so that at the level of cycles

H = θ_{null}^{4} + H_{1},

where [H_{1}] = 27 · 49 λ_{1}^{2}. We have the following characterization of H_{1}:

Theorem 0.6. The locus H_{1} is unirational and its general point corresponds to a Prym variety P (C, η), where (C, η) ∈ R_{6} is a Prym curve such that η ∈ W_{4}(C) − W_{4}^{1}(C) and K_{C} ⊗ η is very ample.

As an application of this circle of ideas, we determine the slope of \overline{A}_{5}. Let \overline{A}_{g} be the perfect cone (first Voronoi) compactification of A_{g} — this is the toroidal compactification of A_{g} constructed using the first Voronoi (perfect) fan decomposition of the cone of semi-positive definite quadratic forms with rational nullspace (see e.g. [Vor1908] for the origins, and [SB06] for recent progress). The Picard group of \overline{A}_{g} with rational coefficients has rank 2 (for g ≥ 2), and it is generated by the first Chern class λ_{1} of the Hodge bundle and the class of the irreducible boundary divisor D := \overline{A}_{g} − A_{g}. The slope of an effective divisor E ∈ Eff(\overline{A}_{g}) is defined as the quantity

s(E) := inf \{\frac{a}{b} : a, b > 0, aλ_{1} − b[D] − [E] = c[D], c > 0\} .

If E is an effective divisor on \overline{A}_{g} with no component supported on the boundary and [E] = aλ_{1} − bD, then s(E) := \frac{a}{b} ≥ 0. One then defines the slope (of the effective cone) of the moduli space as s(\overline{A}_{g}) := inf_{E∈Eff(\overline{A}_{g})} s(E). This important invariant governs to a large extent the birational geometry of \overline{A}_{g}; for instance \overline{A}_{g} is of general type if s(\overline{A}_{g}) < g + 1, and \overline{A}_{g} is uniruled when s(\overline{A}_{g}) > g + 1. Any effective divisor class calculation on \overline{A}_{g} provides an upper bound for s(\overline{A}_{g}). It is known [SM92] that s(\overline{A}_{4}) = 8 (and the minimal slope is computed by the divisor \overline{J}_{4} of Jacobians). In the next case g = 5, the class of the closure of the Andreotti-Mayer divisor is [\overline{N′_{0}}] = 108λ_{1}−14D, giving the upper bound s(\overline{A}_{5}) ≤ \frac{54}{7}.

Theorem 0.7. ^{1} The slope of \overline{A}_{5} is computed by \overline{N′_{0}}, that is, s(\overline{A}_{5}) = \frac{54}{7}. Furthermore, the Kodaira-Iitaka dimension of \overline{N′_{0}} is submaximal, that is, κ(\overline{A}_{5}, \overline{N′_{0}}) < dim(\overline{A}_{5}).

To prove this result, we define a partial compactification R̃_{6} of R_{6} and via the (rational) Prym map P : R̃_{6} −−→ \overline{A}_{5} we investigate the pull-back

P^{∗}(\overline{N′_{0}}) = 2Q̃ + Ũ + 20δ0^{′′},

1Added in April 2022: The published version of the paper asserts a slightly stronger version of this result, which however does not follow from the arguments presented in this paper. This is the corrected result.

where Q̃ and Ũ denote the closure of Q and U respectively in R̃_{6}, and δ0^{′′} is the divisor of degenerate Wirtinger double covers (see Section 6 for precise definitions). Since each of the divisors appearing in this linear system admits a uniruled parametrization in terms of plane sextics having a totally tangent conic, we are ultimately able to establish that \overline{N′_{0}} is an extremal effective divisor on \overline{A}_{5}.

A final application concerns the divisor in \overline{A}_{5} of Pryms obtained from branched covers. The Prym variety associated to a double cover f : C̃ → C branched over two points is still a ppav. When g(C) = 5 (and only in this case), the Prym varieties constructed in this way form an irreducible divisor D^{ram} := P_{∗}(Δ_{0}^{ram}) inside the moduli space. We have the following formula for the class of the closure of D^{ram} in \overline{A}_{5}:

Theorem 0.8. [\overline{D^{ram}}] = 12 (153λ_{1} − 19D) ∈ CH^{1}(\overline{A}_{5}).

Since the classes [P^{∗}(\overline{D^{ram}})] and δ_{0}^{ram} are not proportional, one obtains that the general Prym variety (A, Θ) ∈ D^{ram} obtained from a ramified cover C̃ → C (with g(C) = 5 and g(C̃) = 10), is also the Prym variety induced by an étale cover C̃_{1} → C_{1} (with g(C_{1}) = 6 and g(C̃_{1}) = 11).

We summarize the structure of the paper. The cycle structure of H and θ_{null}^{g−1} is described in Section 2, whereas the classes [θ_{null}^{g−1}], [H] ∈ CH^{2}(A_{g}) are computed in Section 3. The particular case g = 4 is treated in Section 4. After some background on singularities of Prym theta divisors (Section 5), the different geometric realizations of the ramification and antiramification divisors of the Prym map P : R_{6} → A_{5}, as well as the corresponding class calculations on R̃_{6} are presented in Sections 6 and 7. A proof of Theorem 0.7, thus determining the slope of \overline{A}_{5} is given in Section 8. The final sections of the paper are devoted to a complete geometric description in terms of Pryms of the two components of the cycle H in genus 5, see Theorems 0.5 and 0.6. We close by expressing our thanks to the referee for the many pertinent comments which clearly improved the presentation of the paper.

1. Theta divisors and their singularities

In this section we recall notation, definitions, as well as some results from [GSM08]. We denote ℍ_{g} the Siegel upper half-space, i.e. the set of symmetric complex g × g matrices τ with positive definite imaginary part. If σ =

(\begin{matrix}a & b \\ c & d \end{matrix}) ∈ Sp(2g, Z) is a symplectic matrix in g×g block form, then its action

on τ ∈ ℍ_{g} is defined by σ · τ := (aτ + b)(cτ + d)^{−1}, and the moduli space of complex principally polarized abelian variety (ppav for short) is the quotient A_{g} = ℍ_{g}/ Sp(2g, Z), parameterizing pairs (A_{τ} , Θ_{τ}) with A_{τ} = C^{g}/Z^{g}τ +Z^{g}, an abelian variety and Θ_{τ} the (symmetric) polarization bundle. We denote by A_{τ} [2] the group of two-torsion points of A_{τ}. Let ε, δ ∈ (Z/2Z)^{g}, thought of as vectors of zeros and ones; then x = τε/2 + δ/2 ∈ A_{τ} [2], and the shifted bundle t_{x}^{∗}Θ is still a symmetric line bundle. Up to a multiplicative constant the unique section of the above bundle is given by the theta function with characteristic [ε, δ] defined by

θ [\begin{matrix}ε \\ δ \end{matrix}] (τ, z) := ∑_{m∈Z^{g}} exp πi [^{t}(m + \frac{ε}{2})τ(m + \frac{ε}{2}) + 2 ^{t}(m + \frac{ε}{2})(z + \frac{δ}{2})] .

We shall write θ(τ, z) for the theta function with characteristic [0, 0]. The zero scheme of θ(τ, z), as a function of z ∈ A_{τ}, defines the principal polarization Θ_{τ} on A_{τ}.

Theta functions satisfy the heat equation

\frac{∂^{2}θ [\begin{matrix}ε \\ δ \end{matrix}] (τ, z)}{∂z_{j}∂z_{k}} = 2πi(1 + δ_{j,k})\frac{∂θ [\begin{matrix}ε \\ δ \end{matrix}] (τ, z)}{∂τ_{jk}},

(where δ_{j,k} is Kronecker’s symbol).

The characteristic [ε, δ] is called even or odd corresponding to whether the scalar product ^{t}εδ ∈ Z/2Z is zero or one. Consequently, depending on the

characteristic, θ [\begin{matrix}ε \\ δ \end{matrix}] (τ, z) is even or odd as a function of z. A theta constant

is the evaluation at z = 0 of a theta function. All odd theta constants of course vanish identically in τ.

A holomorphic function f : ℍ_{g} → C is called a modular form of weight k with respect to a finite index subgroup Γ ⊂ Sp(2g, Z) if

f(σ · τ) = det(cτ + d)^{k}f(τ) ∀τ ∈ ℍ_{g}, ∀σ ∈ Γ,

and if additionally f is holomorphic at all cusps of ℍ_{g}/Γ. Theta constants with characteristics are modular forms of weight \frac{1}{2} with respect to a finite index subgroup Γ_{g}(4, 8) ⊂ Sp(2g, Z). We refer to [Igu72] for a detailed study of theta functions.

We denote by

ϕ : X_{g} = ℍ_{g} × C^{g}/(Sp(2g, Z) ⋊ Z)^{2g}) → A_{g} = ℍ_{g}/ Sp(2g, Z)

the universal family of ppav, and let Θ_{g} ⊂ X_{g} be the universal theta divisor — the zero locus of θ(τ, z). Following Mumford [Mum83], we denote by S := Sing_{vert} Θ_{g} the locus of singular points of theta divisors of ppav: (4)

S = ⋃_{τ∈A_{g}} Sing Θ_{τ} = \{(τ, z) ∈ ℍ_{g} × C^{g} : θ(τ, z) = \frac{∂θ}{∂z_{i}} (τ, z) = 0, i = 1, …, g\}

(computationally, by an abuse of notation, we will often work locally on S, thinking of it as a locus inside the cover ℍ_{g} × C^{g} of X_{g}). It is known that S ⊂ X_{g} is of pure codimension g + 1, and has three irreducible components [CvdG00], denoted S_{null}, S_{dec}, and S′. Here S_{null} denotes the locus of even two-torsion points that lie on the theta divisor, given locally by g + 1 equations (5) S_{null} := {(τ, z) ∈ X_{g} : θ(τ, z) = 0, z = (τε + δ)/2 for some [ε, δ] ∈ (Z/2Z)_{even}^{2g}} .

To define S_{dec}, recall that a ppav is called decomposable if it is isomorphic to a product of lower-dimensional ppav. We denote then

(6)  S_{dec} := S ∩ ϕ^{−1}(A_{1} × A_{g−1}).

Since the theta divisor of a product (A_{1}, Θ_{1}) × (A_{2}, Θ_{2}) is given by the union (Θ_{1} × A_{2}) ∪ (A_{1} × Θ_{2}), its singular locus contains Θ_{1} × Θ_{2} and is of codimension 2 (see the work [EL97] of Ein and Lazarsfeld for a proof of a conjecture [ADC84] of Arbarello and De Concini that N_{g−2} is in fact equal to the decomposable locus). Thus the fibers of S_{dec} → A_{1,g−1} are all of dimension g − 2, and the codimension of S_{dec} ⊂ X_{g} is equal to g + 1. (We note that any other locus of products A_{h} × A_{g−h} has codimension h(g − h), and contributes no irreducible component of S).

Finally, S′ is the closure of the locus of singular points of theta divisors of indecomposable ppav that are not two-torsion points. Observe that S_{null}, S′, and S_{dec} all come equipped with an induced structure as determinantal subschemes of ℍ_{g} × C^{g}.

The Andreotti-Mayer divisor is then defined (as a cycle) by

N_{0} := ϕ_{∗}(S) = {τ ∈ A_{g} : Sing Θ_{τ} ≠ ∅}.

It can be shown that N_{0} is a divisor in A_{g}, which has at most two irreducible components, see [Deb92],[Mum83].

The theta-null divisor θ_{null} ⊂ A_{g} is the zero locus of the modular form

F_{g}(τ) := ∏_{m even} θ [\begin{matrix}ε \\ δ \end{matrix}] (τ, 0).

Geometrically, it is the locus of ppav for which an even two-torsion point lies on the theta divisor, and it can be shown that θ_{null} = ϕ_{∗}(S_{null}), viewed as an equality of cycles. Similarly for the other component we have N′_{0} = \frac{1}{2}ϕ_{∗}(S′) (the one half appears because a generic ppav in N′_{0} has two singular points ±x on the theta divisor).

Remark 1.1. The two components of N_{0} are zero loci of modular forms (with some character χ in genus 1 and 2): θ_{null} is the zero locus of the modular form F_{g} of weight 2^{g−2}(2^{g} + 1), while N′_{0} must be the zero locus of some modular form I_{g} of weight g!(g + 3)/4 − 2^{g−3}(2^{g} + 1) (the class, and thus the weight, was computed by Mumford [Mum83]). Unlike the explicit formula for F_{g}, the modular form I_{g} is only known explicitly for g = 4, in which case it is the so called Schottky form [Igu81a],[Igu81b]. Various approaches to constructing I_{g} explicitly were developed in [Yos99],[KSM02].

2. Double points on theta divisors that are not ordinary double points

We shall now concentrate on studying the local structure of a theta divisor near its singular point. For this, we look at the tangent space to S and the map between the tangent spaces.

Proposition 2.1. Let x_{0} = (τ_{0}, z_{0}) be a smooth point of S. Then the map (dϕ)_{x_{0}} : T_{x_{0}}(S) → T_{τ_{0}}(N_{0}) is injective if and only if the Hessian matrix

H(x_{0}) := (\begin{matrix}\frac{∂^{2}θ}{∂z_{1}∂z_{1}} (x_{0}) & … & \frac{∂^{2}θ}{∂z_{1}∂z_{g}} (x_{0}) \\ ⋮ & ... & ⋮ \\ \frac{∂^{2}θ}{∂z_{g}∂z_{1}} (x_{0}) & … & \frac{∂^{2}θ}{∂z_{g}∂z_{g}} (x_{0}) \end{matrix})

has rank g.

Proof. Since the subvariety S ⊂ X_{g} is defined by the g + 1 equations (4), the point x_{0} is smooth if and only if the (\frac{g(g+1)}{2} + g) × (g + 1) matrix

M(τ_{0}, z_{0}) := (\begin{matrix}\frac{∂θ}{∂τ_{11}} & … & \frac{∂θ}{∂τ_{gg}} & \frac{∂θ}{∂z_{1}} & … & \frac{∂θ}{∂z_{g}} \\ \frac{∂^{2}θ}{∂z_{1}∂τ_{11}} & … & \frac{∂^{2}θ}{∂z_{1}∂τ_{gg}} & \frac{∂^{2}θ}{∂z_{1}∂z_{1}} & … & \frac{∂^{2}θ}{∂z_{1}∂z_{g}} \\ ⋮ & ... & ⋮ & ⋮ & ... & ⋮ \\ \frac{∂^{2}θ}{∂z_{1}∂τ_{11}} & … & \frac{∂^{2}θ}{∂z_{1}∂τ_{gg}} & \frac{∂^{2}θ}{∂z_{g}∂z_{1}} & … & \frac{∂^{2}θ}{∂z_{g}∂z_{g}} \end{matrix} )

evaluated at x_{0} = (τ_{0}, z_{0}) has rank g + 1. We compute

M(τ_{0}, z_{0}) = (\begin{matrix}\frac{∂θ}{∂τ_{11}} (x_{0}) & … & \frac{∂θ}{∂τ_{gg}} (x_{0}) & 0 & … & 0 \\ \frac{∂^{2}θ}{∂z_{1}∂τ_{11}} (x_{0}) & … & \frac{∂^{2}θ}{∂z_{1}∂τ_{gg}} (x_{0}) & \frac{∂^{2}θ}{∂z_{1}∂z_{1}} (x_{0}) & … & \frac{∂^{2}θ}{∂z_{1}∂z_{g}} (x_{0}) \\ ⋮ & ... & ⋮ & ⋮ & ... & ⋮ \\ \frac{∂^{2}θ}{∂z_{1}∂τ_{11}} (x_{0}) & … & \frac{∂^{2}θ}{∂z_{1}∂τ_{gg}} (x_{0}) & \frac{∂^{2}θ}{∂z_{g}∂z_{1}} (x_{0}) & … & \frac{∂^{2}θ}{∂z_{g}∂z_{g}} (x_{0}) \end{matrix})

Since the map ϕ is the projection on the first \frac{g(g+1)}{2} coordinates, the proposition follows. □

Remark 2.2. From the heat equation for the theta function it follows that if the Hessian matrix H(x_{0}) has rank g, then x_{0} is a smooth point of S.

We also note that from the product rule for differentiation and the heat equation it follows that the second derivative

\frac{∂^{2}θ [\begin{matrix}ε \\ δ \end{matrix}] (τ, z)}{∂z_{j}∂z_{k}}|_{z=0}

restricted to the locus θ [\begin{matrix}ε \\ δ \end{matrix}] (τ, 0) = 0 is also a modular form for Γ_{g}(4, 8).

Since we have different, easier to handle, local defining equations (5) for S_{null}, we can obtain better results in this case.

Proposition 2.3. A point x_{0} ∈ S_{null} is a smooth point of S_{null} unless \frac{∂θ}{∂τ_{ij}} (x_{0}) = 0 for all 1 ≤ i, j ≤ g. The map (dϕ)_{x_{0}} is injective if and only if the Hessian matrix H(x_{0}) has rank g.

Remark 2.4. If x_{0} = (τ_{0}, z_{0}) is a smooth point of S_{null}, while τ_{0} is singular in θ_{null}, this implies that at least two different theta constants vanish at τ_{0}.

Using the above framework, we get a complete description of the intersection S_{null} ∩ S′, obtaining thus an easier proof of one of the main results of [GSM07].

Proposition 2.5. For x_{0} ∈ S_{null}, the point x_{0} lies in S′ if and only if the rank of H(x_{0}) is less than g.

Proof. If x_{0} ∈ S′ ∩ S_{null}, then it is a singular point in S, hence the rank of H(x_{0}) is less than g by the above proposition. To obtain a proof in the other direction, since z_{0} is a two-torsion point, the matrix M(τ_{0}, z_{0}) appearing in the proof of the proposition above has the form

M(x_{0}) = (\begin{matrix}\frac{∂θ}{∂τ_{11}} (x_{0}) & … & \frac{∂θ}{∂τ_{gg}} (x_{0}) & 0 & … & 0 \\ 0 & … & 0 & \frac{∂^{2}θ}{∂z_{1}∂z_{1}} (x_{0}) & … & \frac{∂^{2}θ}{∂z_{1}∂z_{g}} (x_{0}) \\ ⋮ & ... & ⋮ & ⋮ & ... & ⋮ \\ 0 & … & 0 & \frac{∂^{2}θ}{∂z_{g}∂z_{1}} (x_{0}) & … & \frac{∂^{2}θ}{∂z_{g}∂z_{g}} (x_{0}) \end{matrix})

Hence if the rank of H(x_{0}) is less than g, x_{0} is a singular point of S; thus either it is a singular point of S_{null}, or it lies in the intersection S_{null} ∩ S′. The first case cannot happen for dimensional reasons (the singular locus of S_{null} is codimension at least 2 within S_{null}, see also [CvdG00]), and thus we must have x_{0} ∈ S_{null} ∩ S′. □

Corollary 2.6. Set theoretically we have

ϕ(S_{null} ∩ S′) = θ_{null}^{g−1}.

Remark 2.7. From the previous proof it also follows that Sing S_{null} ⊂ S_{null} ∩ S′.

Our further investigation will consider the subvariety

H := S^{′g−1} := {x_{0} = (τ_{0}, z_{0}) ∈ S′ : rk H(x_{0}) < g} ⊂ X_{g}

(notice that since the derivative of a section of a line bundle is a section of the same bundle when restricted to the zero locus of the section, this is an algebraic subvariety of X_{g}). Note that H, being defined by explicit equations in the (derivatives of) theta functions, comes equipped with a scheme structure. Then we define the pushforward cycle

2H := 2N_{0}^{′g−1} := ϕ_{∗}(H) ⊂ A_{g}

Unlike in the case of the theta-null, H ⊄ Sing S. Indeed, if z_{0} is not a twotorsion point, the condition rk H(x_{0}) < g does not imply that rk M(τ_{0}, z_{0}) < g + 1, as the matrix M at z_{0} does not have as many zero entries as in the theta-null case. Still, we have the set-theoretic inclusions

S_{null} ∩ S′ ⊂ H and θ_{null}^{g−1} ⊂ H.

The locus H is given locally by (g + 2) equations (the g + 1 equations for S′ together with the vanishing of the Hessian determinant), and thus each irreducible component of H has codimension at most g + 2 in X_{g}. However, we note that S_{dec} ⊂ H ⊂ S is an irreducible component of codimension g + 1. We now check that all other irreducible components of H are indeed of expected codimension g + 2. Indeed, we first note that by the results of Ciliberto and van der Geer [CvdG08] the Andreotti-Mayer locus N_{k} (parameterizing ppav whose theta divisor has singular locus of dimension at least k) with 1 ≤ k ≤ g − 3 has codimension at least k + 2 in A_{g}, and thus its preimage in S cannot be an irreducible component of S for dimension reasons. Now for both S′ and S_{null} it is known that generically the singular points of the theta divisors are ordinary double points, and thus H cannot be equal to either of these loci. Finally, by the results of Ein and Lazarsfeld [EL97] the locus N_{g−2} is equal to the locus of indecomposable ppav, and each component A_{h} × A_{g−h} of it has codimension too high, except for h = 1.

The above discussion leads to the following result:

Proposition 2.8. The Andreotti-Mayer locus N_{1} is contained in H.

Proof. Indeed, for τ_{0} ∈ N_{1} we let z(t) ⊂ Sing Θ_{τ} be a curve of singular points such that z(0) = z_{0} is a smooth point of the curve. Differentiating (4) with respect to t, we get g non-zero equations (the derivative of the first one will vanish):

∑_{j=1}^{g} \frac{∂^{2}θ(τ_{0}, z(t))}{∂z_{i}∂z_{j}} \frac{∂z_{j}(t)}{∂t} = 0.

Denoting

v := (\frac{∂z_{1}}{∂t} , …, \frac{∂z_{g}}{∂t} ) |_{t=0}

this means that H(x_{0}) · v = 0, and since by our assumption z_{0} is a smooth point of the curve and thus v ≠ 0, the matrix H(x_{0}) has a kernel, and in particular is not of maximal rank. □

Corollary 2.9. For g ≥ 5 the locus of Jacobians J_{g} is contained in H. Hence for g ≥ 5 set-theoretically θ_{null}^{g−1} ⊊ H.

Proof. Indeed, we have J_{g} ⊂ N_{1} ⊂ H for g ≥ 5. However, since for all g the divisor θ_{null} does not contain J_{g}, we must have J_{g} ⊂ \overline{H \ θ_{null}^{g−1}}. □

3. Class computations in cohomology

In this section we compute the class of the components of the expected dimension of the loci H and H in Chow and cohomology rings (our computation works in both, as we only use Chern classes of vector bundles) of X_{g} and A_{g}, respectively.

Recall that Mumford [Mum83] computed the class of N′_{0} in the Picard group of the partial toroidal compactification of the moduli space A_{g} (the class of θ_{null} is easier, and was computed previously by Freitag [Fre83]). We shall compute the classes of the codimension 2 cycles H and θ_{null}^{g−1} on A_{g}. As a consequence we will obtain a complete description of H in genus 4, rederive some result of [GSM08], and reprove that for g ≥ 5 the locus H has other components besides θ_{null}^{g−1}. Debarre [Deb92, Section 4] computed the class of the intersection θ_{null}∩N′_{0} and used this to show that this intersection is not irreducible. In spirit our computation is similar, though much more involved.

For the universal family ϕ : X_{g} → A_{g} we denote by Ω_{X_{g}/A_{g}} the relative cotangent bundle, by 𝔼 := ϕ_{∗}Ω_{X_{g}/A_{g}} we denote its pushforward — the rank g vector bundle that is called the Hodge bundle. Then the Hodge class λ_{1} := c_{1}(𝔼) is the Chern class of the line bundle of modular forms of weight one on A_{g}.

The basic tool for our computation of pushforwards is the following:

Lemma 3.1. The pushforward under ϕ of powers of the universal theta divisor Θ ⊂ X_{g} can be computed as follows:

ϕ_{∗}([Θ^{k}]) = \begin{cases}0 & if k < g \\ g! & if k = g \\ \frac{(g+1)!}{2}λ_{1} & if k = g + 1 \\ \frac{(g+2)!}{8}λ_{1}^{2} & if k = g + 2 \end{cases}

Proof. The first three cases are consequence of the computation in [Mum83, page 373]. The last case is the next step of the same computation, recalling that c_{2}(𝔼) = λ_{1}^{2}/2. In full generality the pushforwards of the universal theta divisor were computed and studied in [vdG99] (note that the universal theta divisor trivialized along the zero section, that is, the class [Θ]−λ_{1}/2, is used there, and it is shown that ϕ_{∗} (([Θ] − λ_{1}/2)^{k}) = 0 unless k = g). □

Note that the locus S is given as the scheme of zeroes of theta function and its derivatives, i.e. given by zeroes of a section of Ω_{X_{g}/A_{g}}(Θ) ⊗ O_{X_{g}}(Θ) (see [Mum83]). Hence

[N_{0}] = ϕ_{∗} (c_{g} (Ω_{X_{g}/A_{g}}(Θ) ⊗_{O_{Xg}} O_{Θ})) .

Recall now that S^{g−1} ⊂ S is defined by the equation det H(x_{0}) = 0. On S, each second derivative of the theta function is a section of Θ, and the determinant of the Hessian matrix is known (see [GSM07],[dJ10]) to be a section of

O_{X_{g}}(gΘ) ⊗ ϕ^{∗}(det 𝔼)^{⊗2} ⊗ O_{S}.

Using the above formula for the class of S, to get H we will need to compute the pushforward

ϕ_{∗} (c_{g} (Ω_{X_{g}/A_{g}}(Θ)_{|Θ}) · (gΘ + 2ϕ^{∗}λ_{1}))

The computation becomes rather delicate since S^{g−1} is not equidimensional. We set

S_{indec} := \overline{S \ S_{dec}} = S′ ∪ S_{null},

which is then purely of codimension g + 2 in X_{g}, and thus we have

[S_{indec}^{g−1}] = [S_{indec}] · (g[Θ] + 2λ_{1}) ∈ CH^{g+2}(X_{g}).

However, for dimension reasons it turns out that we often do not need to deal with the class of S_{dec}:

Proposition 3.2. For g ≥ 4 we have the equality of codimension 2 classes on A_{g}:

[N_{0}^{g−1}] = [N_{0, indec}^{g−1}] := [ϕ_{∗}(S_{indec}^{g−1})]

Moreover this class can be computed as

[N_{0}^{g−1}] = \frac{g!}{8}(g^{3} + 7g^{2} + 18g + 24)λ_{1}^{2} ∈ CH^{2}(A_{g}).

Proof. The first statement is a consequence of the fact that the map ϕ has (g − 2)-dimensional fiber along S_{dec}, and generically 0-dimensional fibers over S_{indec}. (Note that this is the place in the argument where we are using the assumption g ≥ 4 to ensure that S_{indec}^{g−1} is in fact non-empty, and that the codimension of its image under ϕ is lower than the codimension of A_{1} × A_{g−1}.) We now compute

[N_{0}^{g−1}] = ϕ_{∗} (c_{g} (Ω_{X_{g}/A_{g}}(Θ) · Θ) · (gΘ + 2ϕ^{∗}λ_{1}))

= ϕ_{∗} ((Θ^{g} + Θ^{g−1}ϕ^{∗}λ_{1} + Θ^{g−2}ϕ^{∗}λ_{2} + …) · (gΘ^{2} + 2Θϕ^{∗}λ_{1}))

= ϕ_{∗} (g (Θ^{g+2} + Θ^{g+1}ϕ^{∗}λ_{1} + Θ^{g} \frac{ϕ^{∗}λ_{1}^{2}}{2} ) + (2Θ^{g+1}ϕ^{∗}λ_{1} + 2Θ^{g}ϕ^{∗}λ_{1}^{2}))

= (\frac{g(g + 2)!}{8} + \frac{g(g + 1)!}{2} + \frac{g(g)!}{2} + (g + 1)! + 2g!) λ_{1}^{2}

= \frac{g!}{8}(g^{3} + 7g^{2} + 18g + 24)λ_{1}^{2}.

□

We now compute the class of the locus θ_{null}^{g−1}: recall that a theta constant is a modular form of weight \frac{1}{2}, and that the determinant of the Hessian matrix of θ [\begin{matrix}ε \\ δ \end{matrix}] (τ, z) evaluated at z = 0 is a modular form of weight \frac{g+4}{2}

along the zero locus of θ [\begin{matrix}ε \\ δ \end{matrix}] (τ) (see [GSM08],[dJ10]). We thus get:

Proposition 3.3. For g ≥ 2 we have

[θ_{null}^{g−1}] = (g + 4)2^{g−3}(2^{g} + 1)λ_{1}^{2}.

Proof. Indeed, we have

θ_{null}^{g−1} = \{τ ∈ ℍ_{g} : ∃[ε, δ] even , θ [\begin{matrix}ε \\ δ \end{matrix}] (τ) = det (\frac{∂^{2}θ [\begin{matrix}ε \\ δ \end{matrix}] (τ, z)}{∂z_{j}∂z_{k}} ) |_{z=0} = 0\} ,

Since there are 2^{g−1}(2^{g} + 1) even characteristics, for each of them we get a contribution of λ_{1}/2 (for the zero locus of the corresponding theta constant) times (g + 4)λ_{1}/2 (for the Hessian). □

The proof of Theorem 0.2 comes by subtraction using the class formulas established in Propositions 3.2 and 3.3, while taking into account the relation given in formula (3).

4. The case g = 4

In this section we will work out the situation for genus 4 in detail, eventually proving Theorem 0.3. By the above formulas for g = 4 we have

[θ_{null}^{3}] = 272λ_{1}^{2}; [N_{0}^{3}] = 3 · 272λ_{1}^{2}.

Moreover, going back from N_{0}^{g−1} to H = N_{0}^{′g−1}, we recall that for arbitrary genus by definition we have N_{0}^{g−1} = θ_{null}^{g−1} +2H, and since at the intersection of the two components θ_{null} and N′_{0} the singular points lie on both, we also have that set-theoretically

θ_{null}^{g−1} ⊂ H

As an immediate consequence we obtain:

Proposition 4.1. The following identity holds at the level of codimension two cycles on A_{4}:

N_{0}^{3} = 3θ_{null}^{3}.

Proof. From the formulas above we see that the cycle 2θ_{null}^{3} appears inside 2N_{0}^{′3}, and thus that 3θ_{null}^{3} is a subcycle of N_{0}^{3}. Since the Chern classes are equal and θ_{null}^{3} is equidimensional, we need to rule out the possibility of N_{0}^{3} having an extra lower dimensional component. However, for genus 4 we know geometrically that N′_{0} is the locus of Jacobians. Using Riemann’s Singularity Theorem for genus 4 curves we then see that the period matrix of a Jacobian is in N_{0}^{′3} if and only if its theta divisor is singular at a twotorsion point, i.e. if this Jacobian lies in θ_{null}^{3} (notice that this reproves a result of [GSM08]). □

The proof of Theorem 0.3 is an immediate consequence of the above facts. We can prove something more: let I_{4} be the Schottky modular form of weight 8 defining the Jacobian locus. Let then

det D(I_{4}) := det ( \begin{matrix}\frac{∂I_{4}}{∂τ_{11}} & \frac{1}{2} \frac{∂I_{4}}{∂τ_{12}} & … & \frac{1}{2} \frac{∂I_{4}}{∂τ_{14}} \\ \frac{1}{2} \frac{∂I_{4}}{∂τ_{21}} & \frac{∂I_{4}}{∂τ_{22}} & … & \frac{1}{2} \frac{∂I_{4}}{∂τ_{24}} \\ … & … & … & … \\ \frac{1}{2} \frac{∂I_{4}}{∂τ_{41}} & … & … & \frac{∂I_{4}}{∂τ_{44}} \end{matrix} )

The restriction of this determinant to the zero locus of I_{4} is a modular form of weight 34 = 8 · 4 + 2. By Proposition 2.1 we know that for a point in N′_{0} \ H the matrix D(I_{4}) is proportional to the Hessian matrix H(x_{0}), hence it vanishes exactly along θ_{null}^{3}. The class of the cycle

{I_{4} = det D(I_{4}) = 0}

is 8 · 34 λ_{1}^{2} = 272 λ_{1}^{2}. Thus we obtain

Proposition 4.2. The locus θ_{null}^{3} ⊂ A_{4} is a complete intersection given by

I_{4} = det D(I_{4}) = 0.

We observe that, by Riemann’s Theta Singularity theorem, this is the locus of Jacobians with theta divisor singular at a two-torsion point. Moreover, the form \sqrt{F_{4}} (recall that F_{g} is the product of all even theta constants) is well-defined along the Jacobian locus and it has the same weight, hence we get a different proof of the following result recently obtained by Matone and Volpato [MV11]:

Corollary 4.3. On J_{4} we have the equality \sqrt{F_{4}} = c det D(I_{4}) for some constant c. The locus θ_{null}^{3} can also be given by equations

I_{4} = \sqrt{F_{4}} = 0.

Contrary to the situation in genus 4, for higher genera we know that we have other components, see Corollary 2.9. This fact can also be deduced from our class computation as follows.

Proof of Proposition 0.1. Recall that the statement we are proving is that at the level of effective cycles θ_{null}^{g−1} ⊊ H for any g ≥ 5. We first note that the above discussion for the genus 4 case shows that the cycle-theoretic inclusion holds. Secondly, since we have computed both classes, we see that for g ≥ 5 the class of N_{0}^{g−1} is not equal to 3 times the class of θ_{null}^{g−1}. In fact the growth orders of the degrees of these two classes are respectively

deg θ_{null}^{g−1} ∼ 4^{g−4} deg θ_{null}^{3}; deg N_{0}^{g−1} ∼ \frac{g!}{4!} deg N_{0}^{3},

and one would thus expect many additional components. □

The rest of the paper is devoted to studying the geometry for g = 5 in detail; in this case we will be able to describe all components explicitly, and will also obtain many results describing the classical Prym geometry of the situation.

5. Prym theta divisors and their singularities

While for higher g the geometry of the locus H ⊂ A_{g} appears quite intricate, for g = 5 one can use the Prym map P : R_{6} → A_{5}. We begin by setting the notation and reviewing the basic facts about Prym varieties and their moduli, which will be used throughout the rest of the paper.

Let R_{g} be the moduli space of pairs (C, η) with [C] ∈ M_{g}, and η a nonzero two-torsion point of the Jacobian Pic^{0}(C). We denote by f : C̃ → C the étale double cover induced by η (so the genus of C̃ is equal to 2g − 1), by i : C̃ → C̃ the involution exchanging the sheets of f, and by φ_{K_{C}⊗η} : C → PH^{0}(C, K_{C} ⊗ η)^{∨} the Prym-canonical map. The map φ_{K_{C}⊗η} is an embedding if and only if η ∉ C_{2} − C_{2} (where we denote C_{k} := Sym^{k}(C)).

We recall the definition of the Prym map P : R_{g} → A_{g−1}. Consider the norm map Nm_{f} : Pic^{2g−2}(C̃) → Pic^{2g−2}(C) induced by the double cover f. The even component of the preimage

Nm_{f}^{−1}(K_{C})^{+} := {L ∈ Pic^{2g−2}(C̃) : Nm_{f}(L) = K_{C}, h^{0}(C̃, L) ≡ 0 mod 2}

is then an abelian variety of dimension g −1. Denoting by ΘC̃ ⊂ Pic^{2g−2}(C̃) the Riemann theta divisor, scheme-theoretically we have the following equality ΘC̃|_{Nm_{f}^{−1}(K_{C})^{+}} = 2Ξ, where Ξ is a principal polarization. The Prym variety is defined to be the ppav

P (C, η) := (Nm_{f}^{−1}(K_{C})^{+}, Ξ) ∈ A_{g−1}.

The polarization divisor can be described explicitly following [Mum74]:

Ξ(C, η) := {L ∈ Nm_{f}^{−1}(K_{C})^{+} : h^{0}(C̃, L) > 0}.

A key role in what follows is played by the Prym-Petri map

μ_{L}^{−} : ∧^{2}H^{0}(C̃, L) → H^{0}(C, K_{C} ⊗ η), u ∧ v ↦ u · i^{∗}(v) − v · i^{∗}(u),

where one makes the usual identification H^{0}(C, K_{C} ⊗ η) = H^{0}(C̃, KC̃)^{−} with the (−1) eigenspace under the involution i. Following [Wel85], for (C, η) ∈ R_{g} and r ≥ −1, we define the determinantal locus

V_{r}(C, η) := {L ∈ Nm_{f}^{−1}(K_{C}) : h^{0}(L) ≥ r + 1, h^{0}(L) ≡ r + 1 mod 2}.

For a general Prym curve (C, η) ∈ R_{g}, the map μ_{L}^{−} is injective for every L ∈ Nm_{f}^{−1}(K_{C}), and dim V_{r}(C, η) = g − 1 − \binom{r+1}{2}, see [Wel85].

For a point L ∈ Ξ, we recall the description of the tangent cone T C_{L}(Ξ). Suppose h^{0}(C̃, L) = 2m ≥ 2 and we fix a basis {s_{1} …, s_{2m}} of H^{0}(C̃, L). Consider the skew-symmetric matrix

M_{L} := (μ_{L}^{−}(s_{k} ∧ s_{j}))_{1≤k,j≤2m}

and the pfaffian Pf(L) := \sqrt{det(M_{L})} ∈ Sym^{m} H^{0}(C, K_{C} ⊗ η). Via the identification T_{L}(P (C, η)) = H^{0}(C, K_{C} ⊗ η)^{∨} we have the following result of [Mum74]:

Theorem 5.1. If h^{0}(C̃, L) = 2 then Pf(L) = 0 is the equation of the projectivized tangent space PT_{L}(Ξ). If m ≥ 2 then L ∈ Sing(Ξ) and either Pf(L) ≡ 0, in which case mult_{L}(Ξ) ≥ m + 1, or else, Pf(L) = 0 is the equation of the tangent cone PT C_{L}(Ξ).

Note that one can have L ∈ Sing(Ξ) even when m = 1 and Pf(L) is identically zero, so that the Prym theta divisor Ξ can have two types of singularities, as follows:

Definition 5.2. For a point L ∈ Sing(Ξ), one says that

(1) L is a stable singularity if h^{0}(C̃, L) = 2m ≥ 4,

(2) L is an exceptional singularity if L = f^{∗}(M)⊗OC̃(B), where M ∈ Pic(C) is a line bundle with h^{0}(C, M) ≥ 2 and B is an effective divisor on C.

Let Sing_{f}^{st}(Ξ) = V_{3}(C, η) be the locus of stable singularities and Sing_{f}^{ex}(Ξ) the locus of exceptional singularities. Clearly Sing(Ξ) = Sing_{f}^{st}(Ξ)∪Sing_{f}^{ex}(Ξ). Both these notions depend on the étale double cover f : C̃ → C and are not intrinsic to Ξ. Furthermore, there can be singularities that are simultaneously stable and exceptional. Every singularity of a 4-dimensional theta divisor Ξ can in fact be realized as both a stable and an exceptional singularity in different incarnations of (A, Ξ) ∈ A_{5} as a Prym variety.

For a decomposable vector 0 ≠ u ∧ v ∈ ∧^{2}H^{0}(C̃, L), we set

div(u) := D_{u} + B, div(v) := D_{v} + B,

where D_{u}, D_{v} have no common components and B ≥ 0 is an effective divisor on C. The next lemma is well known, see [ACGH85, Appendix C]:

Lemma 5.3. For 0 ≠ u ∧ v ∈ ∧^{2}H^{0}(C̃, L) the following are equivalent. (1) μ_{L}^{−}(u ∧ v) = 0.

(2) D_{u}, D_{v} ∈ |f^{∗}M| where M ∈ Pic(C) with h^{0}(C, M) ≥ 2.

In such a case we write L = f^{∗}(M)⊗OC̃(B), hence K_{C} = M^{⊗2}⊗O_{C}(f_{∗}(B)), in particular h^{0}(C, K_{C} ⊗ M^{⊗(−2)}) ≥ 1, and the Petri map μ_{0}(M) is not injective. In particular, Sing_{f}^{ex}(Ξ) = ∅ if C satisfied the Petri theorem.

Suppose L ∈ V_{3}(C, η) is a quadratic stable singularity, hence h^{0}(C̃, L) = 4 and Pf(L) ≠ 0. Setting P^{5} := P(∧^{2}H^{0}(L)^{∨}) and P^{g−2} := P(H^{0}(K_{C} ⊗ η)^{∨}), we consider the projectivized dual of the Prym-Petri map

δ := P ((μ_{L}^{−})^{∨}) : P^{g−2} → P^{5}.

The Plücker embedding of the Grassmannian G^{∗} := G(2, H^{0}(L)^{∨}) ⊂ P^{5} is a rank 6 quadric whose preimage Q_{L} := δ^{−1}(G^{∗}) is defined precisely by the Pfaffian Pf(L). Note also that rk(Q_{L}) ≤ rk(μ_{L}^{−}). On the other hand let G := G(2, H^{0}(L)) ⊂ P(∧^{2}H^{0}(L)) be the dual Grassmannian. It is again a standard exercise in linear algebra to show the equivalence

rk(Q_{L}) ≤ 4 ⟺ G ∩ P (Ker(μ_{L}^{−})) ≠ ∅.

For a point L ∈ Sing_{f}^{st}(Ξ) one has mult_{L}(Ξ) = 2 if and only if h^{0}(C, M) ≤ 2 for any line bundle M on C such that h^{0}(C̃, L ⊗ f^{∗}M^{∨}) ≥ 1, see [SV04]. We summarize this discussion as follows:

Proposition 5.4. For a quadratic singularity L ∈ Sing_{f}^{st}(Ξ) the following conditions are equivalent:

(1) rk(Q_{L}) ≤ 4.

(2) G ∩ P (Ker(μ_{L}^{−})) ≠ ∅.

(3) L ∈ Sing_{f}^{st}(Ξ) ∩ Sing_{f}^{ex}(Ξ).

6. Petri divisors and the Prym map in genus 6

This section is devoted to the study of singularities of Prym theta divisors of dimension 4 via the Prym map P : R_{6} → A_{5}.

We review a few facts about the Deligne-Mumford compactification \overline{R}_{g} of R_{g}, and refer to [Don92] and [FL10] for details. The space \overline{R}_{g} is the coarse moduli space associated to the Deligne-Mumford stack \overline{R}_{g} of stable Prym curves of genus g. The geometric points of \overline{R}_{g} correspond to triples (X, η, β), where X is a quasi-stable curve with p_{a}(X) = g, η ∈ Pic(X) is a line bundle of total degree 0 on X such that η_{E} = O_{E}(1) for each smooth rational component E ⊂ X with |E ∩ \overline{X − E}| = 2 (such a component is called exceptional), and β : η^{⊗2} → O_{X} is a sheaf homomorphism whose restriction to any non-exceptional component is an isomorphism. Denoting π : \overline{R}_{g} → \overline{M}_{g} the forgetful map, one has the formula [FL10, Example 1.4]

(7)  π^{∗}(δ_{0}) = δ0^{′} + δ0^{′′} + 2δ_{0}^{ram} ∈ CH^{1}(\overline{R}_{g}),

where δ0^{′} := [Δ0^{′}], δ0^{′′} := [Δ0^{′′}], and δ_{0}^{ram} := [Δ_{0}^{ram}] are boundary divisor classes on \overline{R}_{g} whose meaning we recall. Let us fix a general point [C_{xy}] ∈ Δ_{0} corresponding to a smooth 2-pointed curve (C, x, y) of genus g − 1 and the normalization map ν : C → C_{xy}, where ν(x) = ν(y). A general point of Δ0^{′} (respectively of Δ0^{′′}) corresponds to a stable Prym curve [C_{xy}, η], where η ∈ Pic^{0}(C_{xy})[2] and ν^{∗}(η) ∈ Pic^{0}(C) is non-trivial (respectively, ν^{∗}(η) = O_{C}). A general point of Δ_{0}^{ram} is of the form (X, η), where X := C ∪_{{x,y}} P^{1} is a quasi-stable curve with p_{a}(X) = g, whereas η ∈ Pic^{0}(X) is a line bundle characterized by η_{P}_{1} = O_{P}_{1}(1) and η_{C}^{⊗2} = O_{C}(−x − y).

For 1 ≤ i ≤ [\frac{g}{2}] we have a splitting of the pull-back of the boundary

(8)  π^{∗}(δ_{i}) = δ_{i} + δ_{g−i} + δ_{i:g−i} ∈ CH^{1}(\overline{R}_{g}),

where the boundary classes δ_{i} := [Δ_{i}], δ_{g−i} := [Δ_{g−i}] and δ_{i:g−i} := [Δ_{i:g−i}] correspond to the possibilities of choosing a pair of two-torsion line bundles on a smooth curve of genus i and one of genus g−i, such that the first one, the second one, or neither of the corresponding bundles is trivial, respectively, see [FL10].

Often we content ourselves with working on the partial compactification R̃_{g} := π^{−1}(M_{g} ∪ Δ_{0}) of R_{g}. When there is no danger of confusion, we still denote by δ0^{′′}, δ0^{′′} and δ_{0}^{ram} the restrictions of the corresponding boundary classes to R̃_{g}. Note that CH^{1}(R̃_{g}) = Q⟨λ, δ0^{′}, δ0^{′′}, δ_{0}^{ram}⟩.

The extension of the (rational) Prym map P : \overline{R}_{g} −−→ \overline{A}_{g−1} over the general point of each of the boundary divisors of \overline{R}_{g} is well-understood, see e.g. [Don92]. The Prym map contracts Δ0^{′′} and all boundary divisors π^{∗}(Δ_{i}) for 1 ≤ i ≤ ⌊\frac{g}{2}⌋. The Prym variety corresponding to a general point [C_{xy}, η] ∈ Δ0^{′′} as above is the Jacobian Jac(C) of the normalization. Thus P (Δ0^{′′}) = J_{g−1}. The pullback map P^{∗} on divisors has recently been described in [GSM11]: one has

(9)  P^{∗}(λ_{1}) = λ − \frac{δ_{0}^{ram}}{4} , P^{∗}(D) = δ0^{′}.

Remark 6.1. We sketch an alternative way of deriving the first formula in (9). For each (C, η) ∈ R_{g}, there is a canonical identification of vector bundles T_{P (C,η)}^{∨} = H^{0}(C, K_{C} ⊗ η) ⊗ O_{P (C,η)}. The pull-back P^{∗}(𝔼) of the Hodge bundle can be identified with the vector bundle N_{1} on \overline{R}_{g} with fiber N_{1}(C, η) = H^{0}(C, ω_{C} ⊗ η), over each point (C, η) ∈ \overline{R}_{g} (we skip details showing that this description carries over the boundary as well). Therefore P^{∗}(λ_{1}) = c_{1}(N_{1}) = λ − \frac{1}{4}δ_{0}^{ram}, where we refer to [FL10] for the last formula.

We have seen that for [C̃ →^{f} C] ∈ R_{g} with Sing_{f}^{ex}(Ξ) ≠ ∅, the curve C fails the Petri theorem. Let GP_{g,k}^{1} ⊂ M_{g} denote the Gieseker-Petri locus whose general element is a curve C carrying a globally generated pencil M ∈ W_{k}^{1}(C) with h^{0}(C, M) = 2, such that the multiplication map

μ_{0}(M) : H^{0}(C, M) ⊗ H^{0}(C, K_{C} ⊗ M^{∨}) → H^{0}(C, K_{C})

is not injective. It is proved in [Far05] that for \frac{g+2}{2} ≤ k ≤ g − 1, the locus GP_{g,k}^{1} has a divisorial component. As usual, we denote by M_{g,d}^{r} the locus of curves [C] ∈ M_{g} such that W_{d}^{r}(C) ≠ ∅.

In the case of M_{6} there are two Gieseker-Petri loci, both irreducible of pure codimension 1, described as follows:

• The locus GP_{6,4}^{1} of curves [C] ∈ M_{6} having a pencil M ∈ W_{4}^{1}(C) with h^{0}(C, K_{C} ⊗ M^{⊗(−2)}) ≥ 1. We have the following formula for the class of its closure in \overline{M}_{6}, see [EH87]:

[\overline{GP}_{6,4}^{1}] = 94λ − 12δ_{0} − 50δ_{1} − 78δ_{2} − 88δ_{3} ∈ CH^{1}(\overline{M}_{6}).

• The locus GP_{6,5}^{1} of curves with a vanishing theta characteristic; then

[\overline{GP}_{6,5}^{1}] = 8 (65λ − 8δ_{0} − 31δ_{1} − 45δ_{2} − 49δ_{3}) ∈ CH^{1}(\overline{M}_{6}).

The Prym map P : R_{6} → A_{5} is dominant of degree 27 and its Galois group equals the Weyl group of E_{6}, see [DS81],[Don92]. The differential of the Prym map at the level of stacks

(dP)_{(C,η)} : H^{0}(C, K_{C}^{⊗2})^{∨} → (Sym^{2} H^{0}(C, K_{C} ⊗ η))^{∨}

is the dual of the multiplication map at the level of global sections for the Prym-canonical map φ_{K_{C}⊗η}. Thus the ramification divisor of P is a Cartier divisor on R_{6} supported on the locus

Q := {(C, η) ∈ R_{6} : Sym^{2} H^{0}(C, K_{C} ⊗ η) ⟶^{≇} H^{0}(C, K_{C}^{⊗2})} .

The closure of P (Q) inside A_{5} is the branch divisor of P. At a general point (A, Θ) ∈ P (Q) the fiber of P has the structure of the set of lines on a one-nodal cubic surface, that is, P^{−1}(A, Θ) ∩ Q consists of 6 ramification points corresponding to the 6 lines through the node. The remaining 15 points of P^{−1}(A, Θ) are in correspondence with the 15 lines on the onenodal cubic surface not passing through the node. Since deg(P) = 27 it follows that P has simple ramification and Q is reduced. Donagi [Don92, p. 93] established that Q is irreducible by showing that the monodromy acts transitively on a general fiber of P_{|Q}. We sketch a different proof which uses the irreducibility of the moduli space of polarized Nikulin surfaces. We summarize these results as follows:

Proposition 6.2. Set-theoretically, the branch divisor of the map P is equal to the closure N′_{0} of P (Q) in A_{5}. At the level of cycles, P_{∗}[Q] = 6[N′_{0}].

We turn our attention to the geometry of Q. First we compute the class of its closure in R̃_{6}, then we link it to Prym-Brill-Noether theory:

Theorem 6.3. The ramification divisor Q ⊂ R_{6} is irreducible. The class of its closure Q̃ in R̃_{6} equals

[Q̃] = 7λ − δ0^{′} − \frac{3}{2}δ_{0}^{ram} − c_{δ′′_{0}} δ0^{′′} ∈ CH^{1}(R̃_{6}),

where we have the estimate c_{δ′′_{0}} ≥ 4.

Proof. The irreducibility of Q follows from [FV11, Theorem 0.5], where it is proved that Q can be realized as the image of a projective bundle over the irreducible moduli space F_{6}^{𝔑} of polarized Nikulin K3 surfaces of genus 6. To estimate the class of the closure Q̃ of Q in R̃_{6}, we set up two tautological vector bundles N_{1} and N_{2} over R̃_{6} having fibers

N_{1}(X, η) := H^{0}(X, ω_{X} ⊗ η) and N_{2}(X, η) := H^{0}(X, ω_{X}^{⊗2} ⊗ η^{⊗2})

over a point (X, η) ∈ R̃_{6}. There is a morphism ϕ : Sym^{2}(N_{1}) → N_{2} between vector bundles of the same rank given by multiplication of Prym-canonical forms, and we denote by Z the degeneracy locus of ϕ. Using [FL10, Proposition 1.7] we have the following formulas in CH^{1}(R̃_{6})

c_{1}(N_{1}) = λ − \frac{1}{4}δ_{0}^{ram} and c_{1}(N_{2}) = 13λ − δ0^{′} − δ0^{′′} − 3δ_{0}^{ram},

thus [Z] = c_{1}(N_{2})−6c_{1}(N_{1}) = 7λ−δ0^{′}−δ0^{′′}−\frac{3}{2}δ_{0}^{ram}. By definition, Q = Z∩R_{6}.

Furthermore, ϕ is non-degenerate at a general point of Δ0^{′} and Δ_{0}^{ram}, hence the difference Z − Q̃ is an effective divisor supported only on Δ0^{′′}.

Assume now that (X, η) ∈ Δ0^{′′} is a generic point corresponding to a normalization map ν : C → X, where [C, x, y] ∈ M_{5,2} and x, y ∈ C are distinct points such that ν(x) = ν(y). Since ν^{∗}(η) = O_{C}, we obtain an identification H^{0}(X, ω_{X} ⊗ η) = H^{0}(C, K_{C}) whereas H^{0}(X, ω_{X}^{⊗2} ⊗ η^{⊗2}) is a codimension one subspace of H^{0}(C, K_{C}^{⊗2}(2x + 2y)) described by a residue condition at x and y. It is straightforward to check that the kernel

Ker ϕ(X, η) = Ker {Sym^{2} H^{0}(C, K_{C}) → H^{0}(C, K_{C}^{⊗2})}

has dimension 3. Thus [Z] − [Q̃] − 3δ0^{′′} is effective supported on Δ0^{′′}, which implies that c_{δ′′_{0}} ≥ 4. □

Remark 6.4. We shall prove later that in fact c_{δ′′_{0}} = 4.

Even though the locus Q is defined in terms of syzygies of Prym-canonical curves, its points have a characterization in terms of stable singularities of Prym theta divisors.

Theorem 6.5. The theta divisor of a Prym variety P (C, η) ∈ A_{5} has a stable singularity if and only if P ramifies at the point (C, η), that is,

Q = {(C, η) ∈ R_{6} : Sing_{(C,η)}^{st}(Ξ) ≠ ∅} .

Proof. Let us denote by W := {(C, η) ∈ R_{6} : V_{3}(C, η) ≠ ∅} the Prym-Brill-Noether locus corresponding to stable singularities of Prym theta divisors. Our aim is to show that W = Q; we begin by establishing the inclusion W ⊂ Q. First note that if [C] ∈ M_{6} is trigonal, for any two-torsion point η ∈ Pic^{0}(C)[2] − {O_{C}} we can write K_{C} ⊗ η = A ⊗ A′, where A ∈ W_{3}^{1}(C) and A′ ∈ W_{7}^{1}(C). This implies that (C, η) ∈ Q.

Fix now (C, η) ∈ R_{6} and a line bundle L ∈ V_{3}(C, η). If h^{0}(C̃, L) ≥ 6, then C̃ (and hence C as well) must be hyperelliptic, so (C, η) ∈ Q by the previous remark. We may thus assume that h^{0}(C̃, L) = 4 and consider the associated Pfaffian quadric Q_{L} ∈ Sym^{2} H^{0}(C, K_{C} ⊗ η). If Q_{L} ≠ 0, then it contains the Prym-canonical model φ_{K_{C}⊗η}(C), in particular (C, η) ∈ Q. If Q_{L} ≡ 0, then there exists M ∈ Pic(C) with h^{0}(C, M) ≥ 3 and an effective divisor D on C̃, such that L = f^{∗}(M) ⊗ OC̃(B). If deg(M) ≤ 4 then C is hyperelliptic, hence (C, η) ∈ Q. If deg(M) = 5, then B = 0 and C is a smooth plane quintic such that h^{0}(C, M ⊗ η) = 1. It is known, see [Don92, Section 4.3], that in this case P (C, η) is the intermediate Jacobian of a cubic threefold and the differential (dP)_{(C,η)} has corank 2, thus once more (C, η) ∈ Q.

Therefore W ⊂ Q. We claim that W has at least a divisorial component, which follows by exhibiting a point (C, η) ∈ R_{6} and a line bundle L ∈ V_{3}(C, η) such that μ_{L}^{−} is surjective. Assuming this for a moment, we conclude that W = Q by invoking the irreducibility of Q.

To finish the proof we use a realization of Prym curves (C, η) ∈ R_{6} with V_{3}(C, η) ≠ ∅ resembling [FV11, Section 2]. For a line bundle L ∈ V_{3}(C, η) with h^{0}(C̃, L) = 4, if μ_{L}^{+} : Sym^{2} H^{0}(C̃, L) → H^{0}(C, K_{C}) denotes the i^{∗}- invariant part of the Petri map, one has the following commutative diagram:

\xymatrix{C̃ \ar[r]^{(L,i^{∗}L)} \ar[dd]_{f} & P^{3} × P^{3} \ar[ddr]_{q} \ar@{-->}[drr] \\  &  &  & P^{15} = P (H^{0}(L)^{∨} ⊗ H^{0}(L)^{∨}) \ar@{-->}[dl] \\ C \ar[rr]^{μ_{L}^{+}} &  & P^{9} = P(Sym^{2} H^{0}(L)^{∨})}

In this diagram q : P^{3} × P^{3} → P^{9} is the map a ⊗ b ↦ a ⊗ b + b ⊗ a into the projective space of symmetric tensors. Reversing this construction, if ι ∈ Aut(P^{3} × P^{3}) denotes the involution interchanging the two factors, the complete intersection of P^{3} × P^{3} with 4 general ι^{∗}-invariant hyperplanes in H^{0}(I_{P^{3}×P^{3}}(1, 1))^{+} and one general ι^{∗}-anti-invariant hyperplane in H^{0}(I_{P^{3}×P^{3}}(1, 1))^{−} is a smooth curve C̃ ⊂ P^{3} × P^{3}; the automorphism ι_{|C} induces a double cover f : C̃ → C such that Ker(μ_{L}^{−}) has 1-dimensional kernel corresponding to the unique element in H^{0}(I_{P^{3}×P^{3}}(1, 1))^{−}. For more details on this type of argument, we refer to [FV11]. □

7. The antiramification divisor of the Prym map

In this section we describe geometrically the antiramification divisor U of the map P : R_{6} → A_{5}, defined via the equality of divisors

(10)  P^{∗}(N′_{0}) = 2Q + U.

For a general curve [C] ∈ GP_{6,4}^{1}, if M ∈ W_{4}^{1}(C) denotes the pencil such that μ_{0}(M) is not injective, we let x + y ∈ C_{2} be the support of the unique section of K_{C} ⊗ M^{⊗(−2)}. We consider the four line bundles

L_{u,v} := f^{∗}M ⊗ OC̃(x_{u} + y_{v}) ∈ Nm_{f}^{−1}(K_{C}),

where 1 ≤ u, v ≤ 2 and f(x_{u}) = x, f(y_{v}) = y. Using the parity flipping lemma of [Mum74], exactly two of the quantities h^{0}(C̃, L_{u,v}) are equal to 2, the other being equal to 3, that is, Sing_{f}^{ex}(Ξ) contains at least two points. Hence π^{∗}(GP_{6,4}^{1}) ⊂ U. Using Theorem 6.3, equality (10), and the formula for [GP_{6,4}^{1}] ∈ CH^{1}(M_{6}), we compute

[U] = P^{∗}([N′_{0}]) − 2[Q] = 108λ − 14λ = π^{∗}([GP_{6,4}^{1}]) ∈ CH^{1}(R_{6}).

Since the λ-coefficient of any non-trivial effective divisor class on R_{6} must be strictly positive, we obtain the following result:

Proposition 7.1. We have the following equality of divisors on R_{6}:

U = π^{∗}(GP_{6,4}^{1}).

We now determine the pull-back of \overline{N′_{0}} under the map P : R̃_{6} −−→ \overline{A}_{5}. As usual, Ũ denotes the closure of U inside R̃_{6}.

Theorem 7.2. We have the following equality of divisors on R̃_{6}:

P^{∗}(\overline{N′_{0}}) = 2Q̃ + Ũ + 20Δ0^{′′}.

Proof. We use the formula [\overline{N′_{0}}] = 108λ_{1} − 14D, as well as Theorem 6.3 and formula (9) in order to note that the effective class P^{∗}(\overline{N′_{0}})−2\overline{Q}−π^{∗}(\overline{GP}_{6,4}^{1}) is supported only on the boundary divisor Δ0^{′′}.

We now prove that the multiplicity of Δ0^{′′} in P^{∗}(\overline{N′_{0}}) equals 20, or equivalently mult_{Δ′′_{0}} (P^{∗}(\overline{N_{0}})) = 40, since P (Δ0^{′′}) = J_{5} ⊈ θ_{null}. Let Ã_{5} := Bl_{J_{5}}(A_{5})

be the blowup of A_{5} along the Jacobian locus and denote by E ⊂ Ã_{5} the exceptional divisor. Then E is a P^{2}-bundle over J_{5} with the fiber over a point (Jac(C), Θ_{C}) ∈ J_{5} being identified with the space P(I_{2}(K_{C})^{∨}) of pencils of quadrics containing the canonical curve C ⊂ P^{4}. One can lift the Prym map to a map P̃ : R̃_{6} −−→ Ã_{5} by setting for a general point (C_{xy}, η) ∈ Δ0^{′′}

P̃(C_{xy}, η) := ((Jac(C), Θ_{C}), q_{xy}) ∈ Ã_{5},

where q_{xy} ∈ P(I_{2}(K_{C})^{∨}) is the pencil of quadrics containing the union C ∪ ⟨x, y⟩ ⊂ P^{4}. Furthermore, P̃^{∗}(E) = Δ_{0}^{ram}, showing that

mult_{Δ′′_{0}} P^{∗}(\overline{N_{0}}) = mult_{J_{5}}(N_{0}).

To estimate the latter multiplicity we consider a general one-parameter family j : U → A_{5} from a disc U ∋ 0 such that j(0) = (Jac(C), Θ_{C}), with [C] ∈ M_{5} being a general curve. Let Θ_{U} := U ×_{A_{5}} Θ → U be the relative theta divisor over U. The image of the differential (dj)_{0}(T_{0}(U)) can be viewed as a hyperplane h ⊂ P (Sym^{2} H^{0}(K_{C})). The variety Θ_{U} has ordinary double points at those points (0, L) ∈ Θ_{U} where L ∈ Sing(Θ_{C}) = W_{4}^{1}(C) is a singularity such that its tangent cone Q_{L} ∈ PI_{2}(K_{C}) belongs to h. Since the assignment W_{4}^{1}(L) ∋ L ↦ Q_{L} ∈ PI_{2}(K_{C}) is an unramified double cover over a smooth plane quintic, we find that Θ_{U} has 10 nodes. Using the theory of Milnor numbers for theta divisors as explained in [SV85] we obtain

mult_{J_{5}}(N_{0}) = χ(θ_{gen}) − χ(W_{4}(C)) + 10,

where χ(θ_{gen}) = 5! = 120 is the topological Euler characteristic of a general (smooth) theta divisor of genus 5. We finally determine χ(W_{4}(C)), using the resolution C_{4} → W_{4}(C). From the Macdonald formula, see [ACGH85], χ(C_{4}) = (−1)^{g−1}\binom{2g−2}{g−1}_{|g=5} = 70, whereas χ(W_{4}^{1}(C)) = −20,

because g(W_{4}^{1}(C)) = 11. Therefore χ(C_{4}^{1}) = 2χ(W_{4}^{1}(C)) = −40. We find that χ(W_{4}(C)) = χ(C_{4}) − χ(C_{4}^{1}) + χ(W_{4}^{1}(C)) = 90, thus mult_{J_{5}}(N_{0}) = 120 − 90 + 10 = 40. □

Corollary 7.3. We have the following formula in CH^{1}(R̃_{6}):

[Q̃] = 7λ − δ0^{′} − 4δ0^{′′} − \frac{3}{2}δ_{0}^{ram}.

We exploit the geometry of the ramification and antiramification divisors of the Prym map and determine the pushforward of divisor classes on \overline{R}_{6}:

Theorem 7.4. The pushforwards of tautological divisor classes via the rational Prym map P : \overline{R}_{6} −−→ \overline{A}_{5} are as follows:

P_{∗}(λ) = 18 · 27λ_{1} − 57D, P_{∗}(δ_{0}^{ram}) = 4(17 · 27λ_{1} − 57D),

P_{∗}(δ0^{′}) = 27D, P_{∗}(δ0^{′′}) = P_{∗}(δ_{i}) = P_{∗}(δ_{i:g−i}) = 0 for 1 ≤ i ≤ g − 1.

We point out that even though P is not a regular map, it can be extended in codimension 1 such that P is the morphism induced at the level of coarse moduli spaces by a proper morphism of stacks, see e.g. [Don92] p.63-64. Furthermore P^{−1} contracts no divisors, in particular, we can pushforward divisors under P and use the push-pull formula. Perhaps the most novel aspect of Theorem 7.4 is the calculation of the class of the divisor P_{∗}(Δ_{0}^{ram}) consisting of Prym varieties corresponding to ramified double covers C̃ → C of genus 5 curves with two branch points.

Proof. We write the following formulas in CH^{1}(\overline{A}_{5}):

27λ_{1} = P_{∗}P^{∗}λ_{1} = P_{∗}(λ) − \frac{1}{4}P_{∗}(δ_{0}^{ram}),

6 · (108λ_{1} − 14D) = 6[\overline{N′_{0}}] = P_{∗}([\overline{Q}]) = 7P_{∗}(λ) − \frac{3}{2}P_{∗}(δ_{0}^{ram}) − P_{∗}(δ0^{′}).

From [GSM11] it follows that P_{∗}(δ0^{′}) = 27D, whereas obviously P_{∗}(δ0^{′′}) = 0, which suffices to solve the system of equations for coefficients. □

8. The slope of \overline{A}_{5}

Using the techniques developed in previous chapters, we determine the slope of the perfect cone compactification \overline{A}_{5} of A_{5} (note also that by the appendix by K. Hulek to [GSM11] this slope is the same for all toroidal compactification). We begin with some preliminaries. Let D be a Q-divisor on a normal Q-factorial variety X. We say that D is rigid if |mD| = {mD} for all sufficiently large and divisible integers m. Equivalently, the Kodaira-Iitaka dimension κ(X, D) equals zero.

We denote by B(D) := ⋂_{m} Bs(|mD|) the stable base locus of D. We say that D is movable if codim B(D) ≥ 2.

Recall that one defines the slope of \overline{A}_{g} as s(\overline{A}_{g}) := inf_{E∈Eff(\overline{A}_{g})} s(E). In a similar fashion one defines the moving slope of \overline{A}_{g} as the slope of the cone of moving divisors on \overline{A}_{g}, that is,

s′(\overline{A}_{g}) := inf \{s(E) : E ∈ Eff(\overline{A}_{g}), E is movable\} .

Thus s′(\overline{A}_{g}) measures the minimal slope of a divisor responsible for a nontrivial map from \overline{A}_{g} to a projective variety. It is known that s(\overline{A}_{4}) = 8 [SM92], and as an immediate consequence of the result about the slope of M_{4} we have that s′(\overline{A}_{4}) = s(\overline{θ}_{null}) = \frac{17}{2}. In the next case, that of dimension g = 5, the formula [\overline{N′_{0}}] = 108λ_{1} − 14D yields the upper bound s(\overline{A}_{5}) ≤ \frac{54}{7}. A lower bound for the slope s(\overline{A}_{5}) was recently obtained in [GSM11].

We shall now prove Theorem 0.7 and establish that

κ(\overline{A}_{5}, \overline{N′_{0}}) < dim(\overline{A}_{5}),

in particular showing that s(\overline{A}_{5}) = \frac{54}{7}. To prove Theorem 0.7 we translate the problem into a question on the linear series |P^{∗}(\overline{N′_{0}})| on \overline{R}_{6}. One can show that each of the components of P^{∗}(\overline{N′_{0}}) is an extremal divisor on \overline{R}_{6}, however their sum could well have positive Kodaira dimension. Of crucial importance is a uniruled parametrization of \overline{Q} using sextics with a totally tangent conic.

We fix general points q_{1}, …, q_{4} ∈ P^{2}, then set S := Bl_{{q_{i}}_{i=1}^{4}}(P^{2}) → P^{2} and denote by {E_{q_{i}}}_{i=1}^{4} the corresponding exceptional divisors. We make the identification P^{15} := |O_{S}(6)(−2 ∑_{i=1}^{4} E_{q_{i}})|, then consider the space of 4-nodal sextics having a totally tangent conic

X := {(Γ, Q) ∈ P^{15} × |O_{S}(2)| : Γ · Q = 2d, where d ∈ (Γ_{reg})_{6}} .

A parameter count shows that X is pure of dimension 14. We define the rational map v : X −−→ R_{6}

v(Γ, Q) := (C, η := ν^{∗} (O_{Γ}(1)(−d))) ∈ R_{6},

where ν : C → Γ is the normalization map. The image v(X) is expected to be a divisor on R_{6}, and we show that this is indeed the case — this construction yields another geometric characterization of points in Q.

Theorem 8.1. The closure of v(X) inside R_{6} is equal to Q, that is, a general Prym curve (C, η) ∈ Q possesses a totally tangent conic.

Proof. We carry out a class calculation on \overline{R}_{6} and the result will be a consequence of the extremality properties of the class [\overline{Q}] ∈ Eff(\overline{R}_{6}). We work on a partial compactification R′_{6} of R_{6} that is even smaller than R̃_{6}.

Let R′_{6} := R_{6}^{0} ∪ π^{−1}(Δ_{0}^{∗}) be the open subvariety of \overline{R}_{6}, where R_{6}^{0} consists of smooth Prym curves (C, η) for which dim W_{6}^{2}(C) = 0 and h^{0}(C, L⊗η) = 1 for every L ∈ W_{6}^{2}(C), whereas Δ_{0}^{∗} ⊂ Δ_{0} is the locus of curves [C_{xy}], where [C] ∈ M_{5} − M_{5,3}^{1} and x, y ∈ C. Observe that codim(R̃_{6} − R′_{6}, R̃_{6}) = 2, in particular we can identify CH^{1}(R′_{6}) and CH^{1}(R̃_{6}). Over the Deligne-Mumford stack R′_{6} of Prym curves coarsely represented by the scheme R′_{6} (observe that R′_{6} is an open substack of R_{6}), we consider the finite cover

σ : 𝔊_{6}^{2} → R′_{6},

where 𝔊_{6}^{2} is the Deligne-Mumford stack that classifies triples (C, η, L), with (C, η) ∈ R′_{6} and L ∈ W_{6}^{2}(C). Note that a curve [C_{xy}] ∈ Δ′_{0} carries no non-locally free sheaves F ∈ \overline{Pic^{6}}(C_{xy}) with h^{0}(C_{xy}, F) ≥ 5, for F would correspond to a 𝔤_{5}^{2} on the normalization C of C_{xy}, a contradiction. The universal curve p : C → 𝔊_{6}^{2} is equipped both with a universal Prym bundle P ∈ Pic(C) and a universal Poincaré line bundle L ∈ Pic(C) such that L_{|p^{−1}(C,η,L)} = L, for any (C, η, L) ∈ 𝔊_{6}^{2}. We form the codimension 1 tautological classes

(11)  𝔞 := p_{∗} (c_{1}(L)^{2}) , 𝔟 := p_{∗} (c_{1}(L) · c_{1}(ω_{p})) ∈ CH^{1}(𝔊_{6}^{2}),

and the sheaves V_{i} := p_{∗}(L^{⊗i}), where i = 1, 2. Both V_{1} and V_{2} are locally free. The dependence of 𝔞 and 𝔟 on the choice of L is discussed in [FL10]. Using the isomorphism CH^{1}(R′_{6}) = CH^{1}(R′_{6}), one can write the following formulas in CH^{1}(R′_{6}), see [FL10, page 776]: (12) σ_{∗}(𝔞) = −48λ+7π^{∗}(δ_{0}), σ_{∗}(𝔟) = 36λ−3π^{∗}(δ_{0}), σ_{∗}(c_{1}(V)) = −22λ+3π^{∗}(δ_{0}).

We also introduce the sheaf E := p_{∗}(P ⊗L). Since R^{1}p_{∗}(P ⊗L) = 0 (this is the point where we use H^{1}(C, L ⊗ η) = 0 for each (C, η, L) ∈ R′_{6}), applying Grauert’s theorem we obtain that E is locally free and via Grothendieck-Riemann-Roch we compute its Chern classes. Taking into account that p_{∗}(c_{1}(P)^{2}) = δ_{0}^{ram}/2 and p_{∗}(c_{1}(L) · c_{1}(P)) = 0, see [FL10, Proposition 1.6], one computes

(13)  c_{1}(E) = λ − \frac{δ_{0}^{ram}}{4} + \frac{𝔞}{2} − \frac{𝔟}{2} ∈ CH^{1}(𝔊_{6}^{2}).

Similarly, by GRR we find that c_{1}(V_{2}) = λ − 𝔟 + 2𝔞.

After this preparation we return to the problem of describing the closure { \atop v(X)} of v(X) in R′_{6}. For a point (C, η, L) ∈ 𝔊_{6}^{2}, the two-torsion point η is

induced by a conic totally tangent to the image of ν : C →^{|L|} Γ ⊂ P^{2}, if and only if the map given by multiplication followed by projection

χ(C, η, L) : H^{0}(C, L ⊗ η) ⊗ H^{0}(C, L ⊗ η) → H^{0}(C, L^{⊗2})/Sym^{2}H^{0}(C, L)

is not an isomorphism. Working over the stack we obtain a morphism of vector bundles over 𝔊_{6}^{2}

χ : E^{⊗2} → V_{2}/Sym^{2}(V_{1}),

such that the class of { \atop v(X)} is (up to multiplicity) equal to

σ_{∗}c_{1}(\frac{V_{2}}{Sym^{2}(V_{1})} − E^{⊗2}) = 35λ − 5(δ0^{′} + δ0^{′′}) − \frac{15}{2}δ_{0}^{ram} = 5[Z],

where we have used both (12) and (13). We recall that the cycle Z was defined in the proof of Theorem 6.3 as a subvariety of the larger space R̃_{6} with the property that Z ∩ R′_{6} = Q ∩ R′_{6}. Thus the class [{ \atop v(X)}] ∈ CH^{1}(R′_{6}) is proportional (up to the divisor class δ0^{′′}) to the class [Q̃]. It is proved in [FV11, Proposition 3.6] that if D is an effective divisor on R_{6} such that [\overline{D}] = α[\overline{Q}] + β δ0^{′′}, then one has the set-theoretic equality D = Q. Thus we conclude that the closure of v(X) in R_{6} is precisely Q. □

Theorem 8.2. ^{2} Through a general point of the ramification divisor \overline{Q} there passes a rational curve R ⊂ \overline{R}_{6} with the following numerical features:

R · λ = 6, R · δ0^{′} = 27, R · δ0^{′′} = 0, R · δ_{0}^{ram} = 10, R · δ_{i} = R · δ_{i:5−i} = 0,

for i = 1, …, 4. In particular R · \overline{Q} = 0 and R · \overline{U} = 0.

Assuming for the moment Theorem 8.2, we explain how it implies Theorem 0.7. Assume that E ∈ Eff(\overline{A}_{5}) with s(E) ≤ s(\overline{N′_{0}}). First note that one can assume that \overline{N′_{0}} ⊈ supp(E), for else, we can replace E by an effective divisor of the form E′ := E − α\overline{N′_{0}} with α > 0 and still s(E′) ≤ s(\overline{N′_{0}}). After rescaling by a positive factor, we can write E ≡ \overline{N′_{0}} − ϵλ_{1} ∈ Eff(\overline{A}_{5}), where ϵ ≥ 0. Clearly we have P^{∗}(E) ∈ Eff(\overline{R}_{6}); observe that since \overline{N′_{0}} is not a component of E, the ramification divisor \overline{Q} cannot be a component of P^{∗}(E) either. Thus R · P^{∗}(E) ≥ 0, that is,

0 ≤ R · P^{∗}(E) = R · (2\overline{Q} + \overline{U} + 20δ0^{′′}) − ϵR · (λ − \frac{δ_{0}^{ram}}{4} ) = −\frac{7ϵ}{2},

which implies ϵ = 0. Thus s(E) = s(\overline{N′_{0}}) and E cannot be a big divisor.

Proof of Theorem 8.2. We retain the notation from Theorem 8.1 and fix a general element (C, η) ∈ Q corresponding to a sextic curve Γ ⊂ P^{2} having nodes at q_{1}, …, q_{4}. From Theorem 8.1 we may assume that there exists a conic Q ⊂ P^{2} such that Q · Γ = 2(p_{1} + ⋯ + p_{6}), where p_{1}, …, p_{6} ∈ Γ_{reg}. Since the points q_{1}, …, q_{4} ∈ P^{2} are distinct and no three are collinear, it follows that [C] ∉ \overline{GP}_{6,4}^{1}, and this holds even when C has nodal singularities.

To construct the pencil R ⊂ \overline{R}_{6}, we reverse this construction and start with a conic Q ⊂ P^{2} and six general points p_{1}, …, p_{6} ∈ Q on it. On the blowup S′ of P^{2} at the 10 points q_{1}, …, q_{4}, p_{1}, …, p_{6}, we denote by {E_{p_{i}}}_{i=1}^{6} and by {E_{q_{j}}}_{j=1}^{4} the exceptional divisors. For 1 ≤ i ≤ 6, let l_{i} ∈ E_{p_{i}} be the point corresponding to the tangent line T_{p_{i}}(Q). If S̃ is the blowup of S′ at l_{1}, …, l_{6}, by slight abuse of notation we denote by E_{p_{i}}, E_{l_{i}} and E_{q_{j}} the exceptional divisors on S̃ (respectively the proper transforms of exceptional divisors on S′). Then dim |OS̃(6) (−2 ∑_{j=1}^{4} E_{q_{j}} − ∑_{i=1}^{6}(E_{p_{i}} + E_{l_{i}}))| = 3, and we choose a general pencil in this linear system. This pencil induces a curve R ⊂ \overline{R}_{6}. Note that the pencil contains one distinguished element t_{0}, consisting of the union of Q and two conics Q_{1} and Q_{2} passing through q_{1}, …, q_{4}.

2Added in April 2022: In the published version, the intersection numbers R · δ_{0}^{ram} and R · δ0^{′} are computed incorrectly. This is corrected here, with every detail provided. These changed intersection numbers are what lead to a slightly weaker version of Theorem 0.7 than in the published version, as per the following discussion.

Considering the pushforward π_{∗}(R) ⊂ \overline{M}_{6}, after a routine calculation we find:

R·λ = 6, R·(δ0^{′} +δ0^{′′} +2δ_{0}^{ram}) = π_{∗}(R)·δ_{0} = 47, π_{∗}(R)·δ_{i} = 0 for i = 1, 2, 3.

In particular, as expected R · \overline{U} = 0. The points in R ∩ Δ_{0}^{ram} correspond to the case when the underlying Prym structure is not locally free, which happens when one of the points p_{i} becomes singular. For each 1 ≤ i ≤ 6, there is one such curve in R.

We are left with the task of determining the multiplicity (R · δ_{0}^{ram})_{t_{0}}. To that end we describe in more detail the Prym structure of the curves in the pencil R. We denote by φ : P^{1} × P^{1} → P^{2} the double cover branched along the conic Q. For a sextic curve Γ ⊂ P^{2} nodal at q_{1}, …, q_{4} and with Γ·Q = 2(p_{1} +⋯+p_{6}), we observe that Γ′ := φ^{−1}(Γ) has nodes at the points in φ^{−1}(q_{1}) ∪ … ∪ φ^{−1}(q_{4}), as well as at φ^{−1}(p_{i}), for i = 1, …, 6. To Γ we associate the étale double cover C′ → C, where C is the normalization of Γ and C′ is the normalization of Γ′. Assume now Γ_{0} := Q ∪ Q_{1} ∪ Q_{2} is the curve corresponding to the point t_{0} ∈ R and set Q ∩ Q_{1} = {z_{1}, …, z_{4}} and Q ∩ Q_{2} = {z′_{1}, …, z′_{4}}. We denote by C_{0} the partial normalization of Γ_{0} at the 4 points of intersection Q_{1}·Q_{2}. Applying stable reduction, the associated double cover C′_{0} → C_{0} has as source curve C′_{0} the union Q′ ∪ Q′_{1} ∪ Q′_{2}, where Q′_{i} is the double cover of Q_{i} ramified over the 4 points in Q · Q_{i}, whereas Q′ is the hyperelliptic genus 3 cover of Q ramified over z_{1}, z′_{1}, …, z_{4}, z′_{4}. Note that Q′_{1} and Q′_{2} are disjoint, hence p_{a}(C′_{0}) = g(Q′) + g(Q′_{1}) + g(Q′_{2}) + 2 · (4 − 1) = 11. Since C′_{0} → C_{0} is ramified over each of the eight nodes z_{1}, z′_{1}, …, z_{4}, z′_{4} of C_{0}, using for instance [FL10, Remark 1.2], it follows that the Prym curve associated to this cover is of the form [X, η, β], where X is the quasi-stable curve obtained from C_{0} by inserting smooth rational components E_{1}, E′_{1}, …, E_{4}, E′_{4} at the points z_{1}, z′_{1}, …, z_{4}, z′_{4} respectively, the line bundle η ∈ Pic^{0}(X) satisfies η_{E_{i}} = O_{E_{i}}(1) and η_{E′_{i}} = O_{E′_{i}}(1),

whereas if X′ := \overline{X \ ⋃_{i=1}^{4}(E_{i} ∪ E′_{i})}, then

η^{⊗2} = O_{X′}(− ∑_{i=1}^{4}(x_{i} + y_{i} + x′_{i} + y′_{i})),

where \{x_{i}, y_{i}\} = E_{i} ∩ \overline{X \ E_{i}} and \{x′_{i}, y′_{i}\} = E′_{i} ∩ \overline{X \ E′_{i}} for i = 1, …, 4.

We denote by ℂ_{τ}^{3g−3} the versal deformation space of [X, η, β] ∈ \overline{R}_{6} and choose local coordinates τ_{1}, …, τ_{3g−3} such that for i = 1, …, 8, the hyperplane (τ_{i} = 0) corresponds to the locus where the exceptional component E_{i} persists. Furthermore, if ℂ_{t}^{3g−3} is the versal deformation space of the stable model C_{0} of X and C → ℂ_{t}^{3g−3} is the universal family, we consider the map ℂ_{τ}^{3g−3} → ℂ_{t}^{3g−3} given by t_{i} = τ_{i}^{2} for i = 1, …, 8 and t_{i} = τ_{i} for 9 ≤ i ≤ 3g−3 = 15. Then the universal family X → ℂ_{τ}^{3g−3} of Prym curves of genus 6 is obtained from the fibre product C′ := C ×_{ℂ_{t}^{3g−3}} ℂ_{τ}^{3g−3} by blowingup the codimension two loci corresponding to the sections (τ_{i} = 0) → C′.

It follows that the intersection multiplicity of R ×_{\overline{R}_{6}} ℂ_{τ}^{3g−3} with the locus (τ_{1} ⋯ τ_{8}) = 0 is equal to 8 and accordingly

(R · δ_{0}^{ram})_{t_{0}} = \frac{1}{2}(R · π^{∗}(δ_{0}))_{t_{0}} = \frac{1}{2}(π_{∗}(R) · δ_{0})_{[C_{0}]} = \frac{8}{2} = 4,

therefore R · δ_{0}^{ram} = 6 + 4 = 10. Observing that t_{0} ∈ R is the only point in the pencil corresponding to a reducible double cover, we also conclude that R · δ0^{′′} = 0, thus R · δ0^{′} = 27. Using Corollary 7.3, we conclude that

R · \overline{Q} = 7R · λ − R · δ0^{′} − \frac{3}{2}R · δ_{0}^{ram} = 42 − 27 − 15 = 0.

□

Remark 8.3. Theorem 0.7 implies that the divisor \overline{N′_{0}} can be contracted via a birational map having \overline{A}_{5} as its source. Especially from the point of view of the Minimal Model Program for \overline{A}_{5}, it would be interesting to find a new compactification of the moduli space of ppav A_{5}^{∗}, and a birational map f : \overline{A}_{5} −−→ A_{5}^{∗} such that f contracts \overline{N′_{0}}.

9. The Prym realization of the components of H

For each irreducible component of H = N_{0}^{′4} in A_{5}, we describe an explicit codimension 2 subvariety of R_{6} which dominates it via the Prym map. As a consequence, we prove that H consists of two irreducible components, both unirational and of dimension 13. We define two subvarieties of R_{6} corresponding to Prym curves (C, η) such that φ_{K_{C}⊗η} lies on a quadric of rank at most 4, cutting a (Petri special) pencil on C. Depending on the degree of this pencil, we denote these loci by Q_{4} and Q_{5} respectively.

Definition 9.1. We denote by Q_{5} the closure in R_{6} of the locus of curves (C, η) ∈ R_{6} such that C carries two vanishing theta characteristics θ_{1}, θ_{2} ∈ W_{5}^{1}(C) with η = θ_{1} ⊗ θ_{2}^{∨}.

Equivalently, K_{C} ⊗η = θ_{1}⊗θ_{2}, which implies that the Prym-canonical model of C lies on a quadric Q ⊂ P^{4} of rank 4, whose rulings induce θ_{1} and θ_{2} respectively.

Definition 9.2. We denote by Q_{4} the closure in R_{6} of the locus of curves (C, η) ∈ R_{6} such that η ∈ W_{4}^{1}(C) − W_{4}(C) and K_{C} ⊗ η is very ample.

Equivalently, K_{C} ⊗ η = A ⊗ A′, where A ∈ W_{4}^{1}(C) and A′ ∈ W_{6}^{1}(C), and then the image φ_{K_{C}⊗η}(C) lies on a quadric Q ⊂ P^{4} of rank at most 4, whose rulings cut out A and A′ respectively.

Remark 9.3. Along the same lines, one can consider the locus Q_{3} of curves (C, η) ∈ R_{6} such that K_{C} ⊗η = A⊗A′, where A ∈ W_{3}^{1}(C) and A′ ∈ W_{7}^{1}(C). Observe that Q_{3} = π^{−1}(M_{6,3}^{1}), where M_{6,3}^{1} is the trigonal locus inside M_{6}. In particular, codim(Q_{3}, R_{6}) = 2. However from the trigonal construction [Don92, Section 2.4], it follows that P (Q_{3}) = J_{5}, that is, P blows-down Q_{3} and thus Q_{3} plays no further role in describing the components of H in A_{5}.

First we show that Q_{4} lies both in the ramification and the antiramification divisor of the Prym map:

Proposition 9.4. Q_{4} ⊆ Q ∩ U.

Proof. We choose a point (C, η) ∈ Q_{4} general in a component of Q_{4} and write η = M ⊗ O_{C}(−D), where M ∈ W_{4}^{1}(C) and D ∈ C_{4} is an effective divisor. Then we compute

h^{0}(C, K_{C} ⊗ η(−D)) = h^{0}(K_{C} ⊗ M^{∨}) = 3,

that is, ℓ := ⟨D⟩ is a 4-secant line to the Prym canonical model φ_{K_{C}⊗η}(C). Moreover ℓ is contained in the rank 4 quadric Q whose rulings cut out on C the pencils M and K_{C} ⊗ η ⊗ M^{∨} respectively. The line ℓ is not contained in a plane of Q belonging to the ruling Λ that cuts out on C the pencil M, for else it would follow that η = 0. Then ℓ is unisecant to the planes in Λ and if d_{M} ∈ |M| is a general element, then ⟨D + d_{M}⟩ is a hyperplane in P^{4}. Thus

K_{C} ⊗ η = O_{C}(d_{M} + D + x + y),

where x, y ∈ C, that is, H^{0}(C, K_{C} ⊗ M^{⊗(−2)}) ≠ 0, and [C] ∈ GP_{6,4}^{1}. □

Proposition 9.5. The locus Q_{4} is unirational and of dimension 13.

Proof. Since Q_{4} ⊂ Q ∩ U, we use the fact that every curve [C] ∈ GP_{6,4}^{1} is a quadratic section of a nodal quintic del Pezzo surface. In the course of proving Theorem 8.2 we observed that a general Prym curve (C, η) ∈ U is characterized by the existence of a totally tangent conic. We show that a similar description carries over to the case of 1-nodal del Pezzo surfaces.

We fix collinear points q_{1}, q_{2}, q_{3} ∈ P^{2}, a general point q_{4} ∈ P^{2}, then denote by ℓ := ⟨q_{1}, q_{2}, q_{3}⟩ ⊂ P^{2}, by S′ := Bl_{{q_{i}}_{i=1}^{4}}(P^{2}) → P^{2} the surface whose

image by the linear system |O_{S′}(2)(− ∑_{i=1}^{4} E_{q_{i}})| is a 1-nodal del Pezzo

quintic. Set P_{S′}^{3} := |O_{S′}(3)(− ∑_{i=1}^{3} E_{q_{i}} − 2E_{q_{4}})|. Note that Aut(S′) = C^{∗}. We consider the 10-dimensional rational variety

V := {(Q, p_{1}, …, p_{5}) : Q ∈ |O_{P}_{2}(2)|, p_{1}, …, p_{5} ∈ Q}

and the rational map p : V −−→ P^{2} given by p((Q, p_{1}, …, p_{5})) := p_{6}, where p_{6} is the residual point of intersection of Q with the unique cubic E ∈ |O_{P}_{2}(3)| passing through q_{1}, …, q_{4}, p_{1}, …, p_{5}. We consider the linear system

P_{(Q,p_{1},...,p_{5})} :={Γ ∈ |O_{S′}(6)(−2 ∑_{i=1}^{4} E_{q_{i}})| : Γ · Q = 2(p_{1} + ⋯ + p_{6})}.

Claim: For a general (Q, p_{1}, …, p_{5}) ∈ V, we have dim P_{(Q,p_{1},...,p_{5})} = 4, that is, the points q_{1}, …, q_{4}, p_{1}, …, p_{6} fail to impose one independent condition on 4-nodal sextic curves.

Since ℓ + Q + P_{S′}^{3} ⊂ P_{(Q,p_{1},...,p_{5})}, to conclude that dim P_{(Q,p_{1},...,p_{5})} ≥ 4, it suffices to find one curve Γ ∈ P_{(Q,p_{1},...,p_{5})} that does not have ℓ as a component. We choose (Q, p_{1}, …, p_{5}) ∈ V general enough that the corresponding cubic E is smooth. Then 2E ∈ P_{(Q,p_{1},...,p_{5})} and obviously ℓ ⊈ 2E. To finish the proof of the claim, we exhibit a point (Q, p_{1}, …, p_{5}) ∈ V such that dim(P_{(Q,p_{1},...,p_{5})}) = 4. We specialize to the case p_{1} ∈ ℓ and let Q_{2} be the conic determined by p_{2}, …, p_{5} and q_{4}. The cubic E must equal ℓ + Q_{2} and p_{6} ∈ ℓ ∩ Q, so that E · Q = p_{1} + ⋯ + p_{6}. Then P_{(Q,p_{1},...,p_{5})} = ℓ + P′, where

P′ := {Y ∈ |O_{S′}(5)(− ∑_{i=1}^{3} E_{q_{i}} − 2E_{q_{4}})| : Y ·Q_{2} = p_{1}+p_{6}+2(p_{2}+p_{3}+p_{4}+p_{5})}.

Because p_{2}, …, p_{5} ∈ P^{2} are general, dim(P′) = 20 − 3 − 3 − 2 − 8 = 4, which completes the proof of the claim.

We now consider the P^{4}-bundle P := {(Q, p_{1}, …, p_{5}, Γ) : Γ ∈ P_{(Q,p_{1},...,p_{5})}}, together with the map u : P −−→ R_{6}, given by

u((Q, p_{1}, …, p_{5}, Γ)) := (C, η := O_{C}(1)(−p_{1} − ⋯ − p_{6})),

where C ⊂ S′ is the normalization of Γ. Then M := O_{C}(2)(− ∑_{i=1}^{4} E_{q_{i}}) ∈ W_{4}^{1}(C) is Petri special and |M ⊗ η| ≅ |O_{S′}(3)(− ∑_{i=1}^{4} E_{q_{i}} − ∑_{j=1}^{6} p_{j})| ≠ ∅, hence u(P) ⊂ Q_{4}. Therefore there is an induced map ū : P// Aut(S′) −−→ Q_{4} between 13-dimensional varieties. Since every curve (C, η) ∈ Q_{4} has a totally tangent conic and can be embedded in S′, it follows that any M ∈ W_{4}^{1}(C) with h^{0}(C, M ⊗ η) ≥ 1 appears in the way described above, which finishes the proof. □

Another distinguished codimension 2 cycle in R_{6} is the locus

Q′_{4} := {(C, η) ∈ R_{6} : η ∈ W_{2}(C) − W_{2}(C)}

of Prym curves (C, η) for which φ_{K_{C}⊗η} fails to be very ample. Writing η = O_{C}(a + b − p − q), with a, b, p, q ∈ C, then M := O_{C}(2a + 2b) ∈ W_{4}^{1}(C) and the 2-nodal image curve φ_{K_{C}⊗η}(C) lies on a pencil of quadrics in P^{4}, thus also on a singular quadric of type (4, 6). We show however, that this quadric is not the projectivized tangent cone of a quadratic singularity L ∈ Sing_{(C,η)}^{st}(Ξ), hence points in Q′_{4} do not constitute a component of P^{−1}(H).

Proposition 9.6. We have Q′_{4} ⊈ U. In particular, all singularities of the Prym theta divisor corresponding to a general point of Q′_{4} are ordinary double points, that is, P (Q′_{4}) ⊈ H.

Proof. Note that Q′_{4} is not contained in U, then use Proposition 5.4. □

Proposition 9.7. The locus Q_{4} dominates via the Prym map the locus H_{1}, that is, \overline{P (Q_{4})} ⊃ H_{1}.

Proof. We start with a point x_{0} = (τ_{0}, z_{0}) ∈ S′, corresponding to a singular point z_{0} ∈ Θ_{τ_{0}} such that rk H(x_{0}) ≤ 4 and x_{0} is a general point of a component of H − θ_{null}. In particular (A_{τ_{0}}, Θ_{τ_{0}}) can be chosen outside any subvariety of A_{5} having codimension at least 3. Since each component of S′ maps generically finite onto N′_{0}, we find a deformation {x_{t} = (τ_{t}, z_{t})}_{t∈T} ⊂ S′, parameterized by an integral curve T ∋ 0, such that for all t ∈ T − {0}, the corresponding theta divisor Θ_{t} has only a pair of singular points, that is, Sing(Θ_{t}) = {±z_{t}}. Since P (Q) is dense in N′_{0}, after possibly shrinking T, we can find a family of triples {(C_{t}, η_{t}, L_{t})}_{t∈T}, such that (C_{t}, η_{t}) ∈ Q for all t ∈ T, while for t ≠ 0 the line bundle L_{t} ∈ V_{3}(C_{t}, η_{t}) corresponds to the singularity z_{t} ∈ Sing(Ξ_{t}). If we set (C, L, η) := (C_{0}, L_{0}, η_{0}), by semicontinuity we obtain that h^{0}(C, L) ≥ 4. Since rk H(L) = rk Q_{L} ≤ 4, it follows from Proposition 5.4 that L ∈ Sing_{(C,η)}^{st}(Ξ) ∩ Sing_{(C,η)}^{ex}(Ξ), which implies that the Prym-canonical line bundle can be expressed as a sum of two pencils. Since L is not a theta characteristic and P (C, η) ∉ J_{5}, we obtain that the Prym-canonical bundle can be expressed as K_{C} ⊗η = A⊗A′, where A ∈ W_{4}^{1}(C). From Proposition 9.6 it follows that K_{C} ⊗ η can be assumed to be very ample, that is, (C, η) ∈ Q_{4}. □

Corollary 9.8. P (Q_{4}) is a unirational component of H, different from θ_{null}^{4}.

9.1. A parametrization of θ_{null}^{4}. Our aim is to find an explicit unirational parametrization of θ_{null}^{4}.

Proposition 9.9. \overline{P (Q_{5})} = θ_{null}^{4}, where the the closure is taken inside A_{5}.

Proof. This proof resembles that of Proposition 9.7. If ϕ : X_{5} → A_{5} denotes the universal abelian variety, recall that we have showed that ϕ_{∗}(S_{null}∩S′) = θ_{null}^{4}. Thus a point (τ, z) ∈ S_{null} ∩ S′ corresponding to a general point (A_{τ} , Θ_{τ}) of a component of θ_{null}^{4} is a Prym variety P (C, η), where (C, η) ∈ Q ∩ U is a Prym curve such that z ∈ Sing(Θ_{τ}) corresponds to a singularity L ∈ Sing_{(C,η)}^{st}(Ξ) ∩ Sing_{(C,η)}^{ex}(Ξ). Then L = f^{∗}(θ_{1}), where θ_{1} ∈ Pic^{5}(C) is a vanishing theta-null. Since h^{0}(C̃, L) = h^{0}(C, θ_{1}) + h^{0}(C, θ_{1} ⊗ η) ≥ 4, we find that θ_{2} := θ_{1} ⊗ η is another theta characteristic, that is, (C, η) ∈ Q_{5}. Therefore θ_{null}^{4} ⊆ \overline{P (Q_{5})}. The reverse inclusion being obvious, we finish the proof. □

We can now complete the proof of Theorem 0.5. We consider the smooth quadric Q := P^{1} × P^{1} and the linear systems of rational curves

P_{1}^{7} := |O_{P^{1}×P^{1}}(3, 1)| and P_{2}^{7} := |O_{P^{1}×P^{1}}(1, 3)| .

Over P_{1}^{7} × P_{2}^{7} we define the P^{5}-bundle

U := {(R_{1}, R_{2}, Γ) : R_{i} ∈ P_{i}^{7} for i = 1, 2, Γ ∈ |I_{R_{1}·R_{2}/Q}^{2}(5, 5)|} .

There is an induced rational map ψ : U −−→ Q_{5} given by

ψ(R_{1}, R_{2}, Γ) := (C, p_{1}^{∗}O(1) ⊗ p_{2}^{∗}O(−1)) ∈ R_{6},

where ν : C → Γ is the normalization map and p_{1}, p_{2} : C → P^{1} are the composition of ν with the two projections.

A general pair (R_{1}, R_{2}) ∈ P_{1}^{7} × P_{2}^{7} corresponds to smooth rational curves such that the intersection cycle R_{1} · R_{2} = o_{1} + ⋯ + o_{10} consists of distinct points. For any curve Γ ∈ |I_{R_{1}·R_{2}}^{2}(5, 5)| we have R_{1} · Γ = R_{2} · Γ = 2(o_{1} + ⋯ + o_{10}). Since ν^{∗} : |I_{R_{1}·R_{2}}(3, 3)| → |K_{C}| is an isomorphism, it follows that both p_{1}^{∗}O(1) and p_{2}^{∗}O(1) are vanishing theta-nulls, hence ψ(U) ⊂ Q_{5}.

Theorem 9.10. The rational map ψ : U −−→ Q_{5} is generically finite and dominant. In particular Q_{5} (and thus θ_{null}^{4} = \overline{P (Q_{5})}) is unirational.

Proof. We start with a point (C, θ_{1}, θ_{2}) ∈ Q_{5} moving in a 13-dimensional family. In particular, the image Γ of the induced map φ_{(θ_{1},θ_{2})} : C → P^{1} ×P^{1} is nodal and we set Sing(Γ) = {o_{1}, …, o_{10}}.

We choose divisors D, D′ ∈ |θ_{1}|, corresponding to lines ℓ, ℓ′ ∈ |O_{Q}(1, 0)| such that ν^{∗}(Γ·ℓ) = D and ν^{∗}(Γ·ℓ′) = D′ respectively. Then D+D′ ∈ |K_{C}|, and since the linear system |I_{o_{1}+···+o_{10}}(3, 3)| cuts out the canonical system on C, it follows that there exists a cubic curve E ∈ |O_{Q}(3, 3)| such that

E · Γ = D + D′ + 2 ∑_{i=1}^{10} o_{i}.

By Bézout’s Theorem, both ℓ and ℓ′ must be components of E, that is, we can write E = ℓ + ℓ′ + R_{1}, where R_{1} ∈ |O_{Q}(1, 3)| is such that R_{1} · Γ = 2 ∑_{i=1}^{10} o_{i}. Switching the roles of θ_{1} and θ_{2}, there exists R_{2} ∈ |O_{Q}(3, 1)| such that R_{2} · Γ = 2 ∑_{i=1}^{10} o_{i}. It follows that (R_{1}, R_{2}, Γ) ∈ ψ^{−1} ((C, θ_{1} ⊗ θ_{2}^{∨})). The variety U being a P^{5}-bundle over P_{1}^{7} × P_{2}^{7} is unirational, hence Q_{5} is unirational as well, thus finishing the proof. □

Acknowledgments: Research of the first author is supported by the Sonderforschungsbereich 647 “Raum-Zeit-Materie” of the DFG. The first author is grateful to INdAM and Università Roma Tre for providing financial support and a stimulating environment during a stay in Rome in which parts of this paper have been completed. Research of the second author was supported in part by National Science Foundation under the grant DMS-10-53313. Research of the third author is supported in part by Università “La Sapienza” under the grant C26A10NABK. Research of the fourth author is supported in part by the research project “Geometry of Algebraic Varieties” of the Italian frame program PRIN-2008.

References

[ACGH85] E. Arbarello, M. Cornalba, P. A. Griffiths, and J. Harris. Geometry of algebraic curves. Vol. I, volume 267 of Grundlehren der Mathematischen Wissenschaften. Springer-Verlag, New York, 1985.

[ADC84] E. Arbarello and C. De Concini. On a set of equations characterizing Riemann matrices. Ann. of Math., 120(1):119–140, 1984.

[AM67] A. Andreotti and A. L. Mayer. On period relations for abelian integrals on algebraic curves. Ann. Scuola Norm. Sup. Pisa (3), 21:189–238, 1967.

[Bea77] A. Beauville. Prym varieties and the Schottky problem. Invent. Math., 41(2):149–196, 1977.

[CvdG00] C. Ciliberto and G. van der Geer. The moduli space of abelian varieties and the singularities of the theta divisor. Surv. Differential Geom., VII, pages 61–81, Int. Press, Somerville, MA, 2000.

[CvdG08] C. Ciliberto and G. van der Geer. Andreotti-Mayer loci and the Schottky problem. Doc. Math., 13:453–504, 2008.

[Deb92] O. Debarre. Le lieu des variétés abéliennes dont le diviseur thêta est singulier a deux composantes. Ann. Sci. École Norm. Sup. (4), 25(6):687–707, 1992.

[Don92] R. Donagi. The fibers of the Prym map. In Curves, Jacobians, and abelian varieties (Amherst, MA, 1990), volume 136 of Contemp. Math., pages 55–125. Amer. Math. Soc., Providence, RI, 1992.

[DS81] R. Donagi and R. Smith. The structure of the Prym map. Acta Math., 146(1-2):25–102, 1981.

[EH87] D. Eisenbud and J. Harris. The Kodaira dimension of the moduli space of curves of genus ≥ 23. Invent. Math., 90:359–387, 1987.

[EL97] L. Ein and R. Lazarsfeld. Singularities of theta divisors and the birational geometry of irregular varieties. J. Amer. Math. Soc., 10(1):243–258, 1997.

[Far05] G. Farkas. Gaussian maps, Gieseker-Petri loci and large theta-characteristics. J. Reine Angew. Math., 581:151–173, 2005.

[FL10] G. Farkas and K. Ludwig. The Kodaira dimension of the moduli space of Prym varieties. J. European Math. Soc. (JEMS), 12(3):755–795, 2010.

[FV11] G. Farkas and A. Verra. Moduli of theta-characteristics via Nikulin surfaces. Math. Annalen, 354:465-496, 2012.

[HF06] H. Farkas. Vanishing thetanulls and Jacobians. In The geometry of Riemann surfaces and abelian varieties, volume 397 of Contemp. Math., pages 37–53. Amer. Math. Soc., Providence, RI, 2006.

[Fre83] E. Freitag. Siegelsche Modulfunktionen, volume 254 of Grundlehren der Mathematischen Wissenschaften. Springer-Verlag, Berlin, 1983.

[vdG99] G. van der Geer. Cycles on the moduli space of abelian varieties, pages 65–89. Aspects Math., E33. Vieweg, Braunschweig, 1999.

[GSM07] S. Grushevsky and R. Salvati Manni. Singularities of the theta divisor at points of order two. Int. Math. Res. Not. IMRN, (15):Art. ID rnm045, 15, 2007.

[GSM08] S. Grushevsky and R. Salvati Manni. Jacobians with a vanishing theta-null in genus 4. Israel J. Math., 164:303–315, 2008.

[GSM11] S. Grushevsky and R. Salvati Manni. The Prym map on divisors, and the slope of A_{5} (with an appendix by K. Hulek). 2011. preprint arXiv:1107.3094.

[Igu72] J.-I. Igusa. Theta functions, volume 194 of Grundlehren der Mathematischen Wissenschaften. Springer-Verlag, New York, 1972.

[Igu81a] J.-I. Igusa. On the irreducibility of Schottky’s divisor. J. Fac. Sci. Univ. Tokyo Sect. IA Math., 28(3):531–545 (1982), 1981.

[Igu81b] J.-I. Igusa. Schottky’s invariant and quadratic forms. In E. B. Christoffel (Aachen/Monschau, 1979), pages 352–362. Birkhäuser, Basel, 1981.

[dJ10] R. de Jong. Theta functions on the theta divisor. Rocky Mountain J. Math., 40(1):155–176, 2010.

[KSM02] J. Kramer and R. Salvati Manni. An integral characterizing the Andreotti-Mayer locus. Abh. Math. Sem. Univ. Hamburg, 72:47–57, 2002.

[Mum74] D. Mumford. Prym varieties. I. In Contributions to analysis (a collection of papers dedicated to Lipman Bers), pages 325–350. Academic Press, New York, 1974.

[Mum83] D. Mumford. On the Kodaira dimension of the Siegel modular variety. In Algebraic geometry—open problems (Ravello, 1982), volume 997 of Lecture Notes in Math., pages 348–375, Berlin, 1983. Springer.

[MV11] M. Matone and R. Volpato. Vector-valued modular forms from the Mumford form, Schottky-Igusa form, product of thetanullwerte and the amazing Klein formula. 2011. preprint arXiv:1102.0006.

[SM92] R. Salvati Manni. Modular forms of the fourth degree. Remark on the paper: “Slopes of effective divisors on the moduli space of stable curves” [Invent. Math. 99 (1990), 321–355] by J. Harris and I. Morrison. Volume 1515 of Lecture Notes in Math., pages 106–111. Springer, Berlin, 1992.

[SB06] N. Shepherd-Barron, Perfect forms and the moduli space of abelian varieties. Invent. Math., 163:25–45, 2006.

[SV85] R. Smith and R. Varley. Components of the locus of singular theta divisors of genus 5. In Algebraic geometry, Sitges (Barcelona), 1983, volume 1124 of Lecture Notes in Math., pages 338–416. Springer, Berlin, 1985.

[SV04] R. Smith and R. Varley. A necessary and sufficient condition for Riemann’s singularity theorem to hold on a Prym theta divisor. Compos. Math., 140(2):447– 458, 2004.

[SV12a] R. Smith and R. Varley. Deformations of isolated even double points of corank one. Proc. Amer. Math. Soc. to appear.

[SV12b] R. Smith and R. Varley. A splitting criterion for an isolated singularity at x = 0 in a family of even hypersurfaces. Manuscripta Math. 137:233–245, 2012.

[Vor1908] G. Voronoi. Nouvelles Applications Des Paramètres continus à la théorie des formes quadratiques. Premier mémoire. Sur quelques propriétés des formes quadratiques positives parfaites. J. Reine Angew. Math. 133:79–178, 1909.

[Wel85] G. Welters. A theorem of Gieseker-Petri type for Prym varieties. Ann. Sci. École Norm. Sup. (4), 18(4):671–683, 1985.

[Yos99] K.-I. Yoshikawa. Discriminant of theta divisors and Quillen metrics. J. Differential Geom., 52(1):73–115, 1999.

Humboldt Universität zu Berlin, Institut für Mathematik, Unter den Linden 6, 10099 Berlin, Germany

   E-mail address: farkas@math.hu-berlin.de

Stony Brook University, Department of Mathematics, Stony Brook, NY 11790-3651, USA

   E-mail address: sam@math.sunysb.edu

Università “La Sapienza”, Dipartimento di Matematica, Piazzale A. Moro 2, I-00185, Roma, Italy

   E-mail address: salvati@mat.uniroma1.it

Università “Roma 3”, Dipartimento di Matematica, Largo San Leonardo Murialdo I-00146 Roma, Italy

   E-mail address: verra@mat.uniroma3.it