THE KODAIRA DIMENSION OF THE MODULI SPACE OF PRYM VARIETIES

GAVRIL FARKAS AND KATHARINA LUDWIG

Prym varieties provide a correspondence between the moduli spaces of curves and abelian varieties M_{g} and A_{g−1}, via the Prym map P_{g} : R_{g} → A_{g−1} from the moduli space R_{g} parameterizing pairs [C, η], where [C] ∈ M_{g} is a smooth curve and η ∈ Pic^{0}(C)[2] is a torsion point of order 2. When g ≤ 6 the Prym map is dominant and R_{g} can be used directly to determine the birational type of A_{g−1}. It is known that R_{g} is rational for g = 2, 3, 4 (see [Dol] and references therein and [Ca] for the case of genus 4) and unirational for g = 5 (cf. [IGS] and [V2]). The situation in genus 6 is strikingly beautiful because P_{6} : R_{6} → A_{5} is equidimensional (precisely dim(R_{6}) = dim(A_{5}) = 15). Donagi and Smith showed that P_{6} is generically finite of degree 27 (cf. [DS]) and the monodromy group equals the Weyl group W E_{6} describing the incidence correspondence of the 27 lines on a cubic surface (cf. [D1]). There are three different proofs that R_{6} is unirational (cf. [D1], [MM], [V]). Verra has very recently announced a proof of the unirationality of R_{7} (see also Theorem 0.8 for a weaker result). The Prym map P_{g} is generically injective for g ≥ 7 (cf. [FS]), although never injective. In this range, we may regard R_{g} as a partial desingularization of the moduli space P_{g}(R_{g}) ⊂ A_{g−1} of Prym varieties of dimension g − 1.

The scheme R_{g} admits a suitable modular compactification \overline{R}_{g}, which is isomorphic to (1) the coarse moduli space of the stack \overline{R}_{g} = \overline{M}_{g}(Bℤ_{2}) of Beauville admissible double covers (cf. [B], [ACV]) and (2) the coarse moduli space of the stack of Prym curves (cf. [BCF]). The forgetful map π : R_{g} → M_{g} extends to a finite map π : \overline{R}_{g} → \overline{M}_{g}. The aim of this paper is to initiate a study of the enumerative and global geometry of \overline{R}_{g}, in particular to determine its Kodaira dimension. The main result of the paper is the following:

Theorem 0.1. The moduli space of Prym varieties \overline{R}_{g} is of general type for g > 13 and g ≠ 15. The Kodaira dimension of \overline{R}_{15} is at least 1.

We point out in Remark 2.9 that the existence of an effective divisor D ∈ Eff(\overline{M}_{15}) of slope s(D) < 6 + 12/(g + 1) = 27/4 (that is, violating the Harris-Morrison Slope Conjecture on \overline{M}_{15}), would imply that \overline{R}_{15} is of general type. There are known examples of divisors D ∈ Eff(\overline{M}_{g}) satisfying s(D) < 6 + \frac{12}{g+1} for every genus of the form g = s(2s + si + i + 1) with s ≥ 2 and i ≥ 0 (cf. [F1], [F2]). No such examples have been found yet on \overline{M}_{15}, though they are certainly expected to exist.

The normal variety \overline{R}_{g} has finite quotient singularities and an important part of the proof is concerned with showing that pluricanonical forms defined on the smooth part \overline{R}_{g}^{reg} ⊂ \overline{R}_{g} can be lifted to any resolution of singularities R̂_{g} → \overline{R}_{g}, that is, we have isomorphisms

Research of the first author partially supported by an Alfred P. Sloan Fellowship, the NSF Grant DMS-0500747 and a Texas Research Assignment.

H^{0}(\overline{R}_{g}^{reg}, K_{\overline{R}_{g}}^{⊗l}) ≅ H^{0}(R̂_{g}, K⊗l ; R̂_{g})

for l ≥ 0. This is achieved in the last section of the paper. The locus of non-canonical singularities in \overline{R}_{g} is also explicitly described: A Prym curve [X, η, β] ∈ \overline{R}_{g} is a noncanonical singularity if and only if X has an elliptic tail C with Aut(C) = ℤ_{6}, such that the line bundle η_{C} ∈ Pic^{0}(C)[2] is trivial (cf. Theorem 6.7).

We outline the strategy to prove that \overline{R}_{g} is of general type for given g. If λ = π^{∗}(λ) ∈ Pic(\overline{R}_{g}) is the pull-back of the Hodge class and δ′_{0}, δ0^{′′}, δ_{0}^{ram} ∈ Pic(\overline{R}_{g}) and δ_{i}, δ_{g−i}, δ_{i:g−i} ∈ Pic(\overline{R}_{g}) for 1 ≤ i ≤ [g/2] are boundary divisor classes such that

π^{∗}(δ_{0}) = δ′_{0} + δ0^{′′} + 2δ_{0}^{ram} and π^{∗}(δ_{i}) = δ_{i} + δ_{g−i} + δ_{i:g−i} for 1 ≤ i ≤ [g/2]

(see Section 2 for a precise definition of these classes), then one has the formula

K_{\overline{R}_{g}} ≡ 13λ − 2(δ0^{′} + δ0^{′′}) − 3δ_{0}^{ram} − 2 ∑_{i=1}^{[g/2]}(δ_{i} + δ_{g−i} + δ_{i:g−i}) − (δ_{1} + δ_{g−1} + δ_{1:g−1}).

We show that this class is big by explicitly constructing effective divisors D on \overline{R}_{g} such that one can write K_{\overline{R}_{g}} ≡ α·λ+β ·D +\{effective combination of boundary classes \}, for certain α, β ∈ ℚ_{>0} (see (2) for the inequalities the coefficients of such D must satisfy).

We carry out an enumerative study of divisors on \overline{R}_{g} defined in terms of pairs [C, η] such that the 2-torsion point η ∈ Pic^{0}(C) is transversal with respect to the theta divisors associated to certain stable vector bundles on C. We fix integers k ≥ 2 and b ≥ 0 and then define the integers

i := kb + k − b − 2, r := kb + k − 2, g := ik + 1 and d := rk.

The Brill-Noether number ρ(g, r, d) := g − (r + 1)(g − d + r) = 0 and a general [C] ∈ M_{g} carries a finite number of line bundles L ∈ W_{d}^{r}(C). For each such line bundle L, if Q_{L} denotes the dual of the Lazarsfeld bundle defined by the exact sequence (see [L])

0 ⟶ Q_{L}^{∨} ⟶ H^{0}(C, L) ⊗ O_{C} ⟶ L ⟶ 0,

we compute that μ(Q_{L}) = d/r = k and then μ(∧^{i}Q_{L}) = ik = g − 1. In these circumstances we define the Raynaud divisor (degeneration locus of virtual codimension 1)

Θ_{∧^{i}Q_{L}} := {η ∈ Pic^{0}(C) : H^{0}(C, ∧^{i}Q_{L} ⊗ η) ≠ 0}.

This is a virtual divisor inside Pic^{0}(C), in the sense that either Θ_{∧^{i}Q_{L}} = Pic^{0}(C) or else Θ_{∧^{i}Q_{L}} is a divisor on Pic^{0}(C) belonging to the linear system |\binom{r}{i}θ|, cf. [R]. We study the relative position of η with respect to Θ_{∧^{i}Q_{L}} and introduce the following locus on \overline{R}_{g}:

D_{g:k} := {[C, η] ∈ R_{g} : ∃L ∈ W_{d}^{r}(C) such that η ∈ Θ_{∧^{i}Q_{L}}}.

When k = 2, i = b, then g = 2i + 1, d = 2g − 2 and D_{2i+1:2} has a new incarnation using the proof of the Minimal Resolution Conjecture [FMP]. In this case, L = K_{C} (a genus g curve has only one 𝔤_{2g−2}^{g−1}!) and [FMP] gives an identification of cycles

Θ_{∧^{i}Q_{K_{C}}} = C_{i} − C_{i} ⊂ Pic^{0}(C),

where the right-hand-side stands for the i-th difference variety of C.

We prove in Section 2 that D_{g:k} is an effective divisor on R_{g}. By specialization to the k-gonal locus M_{g,k}^{1} ⊂ M_{g}, we show that for a generic [C, η] ∈ R_{g} the vanishing H^{0}(C, ∧^{i}Q_{L} ⊗ η) = 0 holds for every L ∈ W_{d}^{r}(C) (Theorem 2.3). Then we extend the determinantal structure of D_{g:k} to a partial compactification of R_{g} which enables us to compute the class of the compactification \overline{D}_{g:k}. Precisely we construct two vector bundles E and F over a stack \overline{R}_{g}^{0} which is a partial compactification of R_{g}, such that rank(E) = rank(F), together with a vector bundle homomorphism ϕ : E → F such that Z_{1}(ϕ) ∩ R_{g} = D_{g:k}. Then we explicitly determine the class c_{1}(F − E) ∈ A^{1}(\overline{R}_{g}^{0}) (Theorem 2.8). The cases of interest for determining the Kodaira dimension of \overline{R}_{g} are when k = 2, 3 when we obtain the following results:

Theorem 0.2. The closure of the divisor D_{2i+1:2} = {[C, η] ∈ R_{2i+1} : h^{0}(C, ∧^{i}Q_{K_{C}} ⊗ η) ≥ 1} inside \overline{R}_{2i+1} has class given by the following formula in Pic(\overline{R}_{2i+1}):

\overline{D}_{2i+1:2} ≡ \frac{1}{2i − 1}\binom{2i}{i}((3i+1)λ−\frac{i}{2}(δ′_{0}+δ0^{′′})−\frac{2i + 1}{4}δ_{0}^{ram}−(3i−1)δ_{g−1}−i(δ_{1:g−1}+δ_{1})−⋯).

To illustrate Theorem 0.2 in the simplest case, i = 1 hence g = 3, we write D_{3:2} = {[C, η] ∈ R_{3} : η = O_{C}(x − y), x, y ∈ C}. The analysis carried out in Section 5 shows that the vector bundle morphism ϕ : E → F is generically non-degenerate along the boundary divisors Δ′_{0}, Δ_{0}^{ram} ⊂ \overline{R}_{3} and degenerate (with multiplicity 1) along the divisor Δ0^{′′} ⊂ \overline{R}_{3} of Wirtinger covers. Theorem 0.2 reads like

\overline{D}_{3:2} ≡ c_{1}(F − E) − δ0^{′′} ≡ 8λ − δ′_{0} − 2δ0^{′′} − \frac{3}{2}δ_{0}^{ram} − 6δ_{1} − 4δ_{2} − 2δ_{1:2} ∈ Pic(\overline{R}_{3}),

and then π_{∗}(\overline{D}_{3:2}) = 56(9λ−δ_{0}−3δ_{1}) = 56·\overline{M}_{3,2}^{1} ∈ Pic(\overline{M}_{3}) (see Theorem 5.1). Theorem 0.2 is consistent with the following elementary fact, see e.g. [HF]: If [C̃ → C] ∈ R_{3} is an étale double cover, then [C̃] ∈ M_{5} is hyperelliptic if and only if [C] ∈ M_{3} is hyperelliptic and η = O_{C}(x − y), with x, y ∈ C being Weierstrass points.

Theorem 0.3. For b ≥ 1 and r = 3b + 1 the class of the divisor \overline{D}_{6b+4:3} on \overline{R}_{6b+4} is given by:

\overline{D}_{g:3} ≡ \frac{4}{r}\binom{6b + 3}{b, 2b, 3b + 3}((3b+2)(b+2)λ−\frac{3b^{2} + 7b + 3}{6}(δ′_{0}+δ0^{′′})−\frac{24b^{2} + 47b + 21}{24}δ_{0}^{ram}−⋯).

Theorems 2.8, 0.2 and 0.3 specify precisely the λ, δ′_{0}, δ0^{′} and δ_{0}^{ram} coefficients in the expansion of [\overline{D}_{g:k}]. Good lower bounds for the remaining boundary coefficients of [\overline{D}_{g:k}] can be obtained using Proposition 1.9. The information contained in Theorems 0.2 and 0.3 is sufficient to finish the proof of Theorem 0.1 for odd genus g = 2i + 1 ≥ 15.

When b = 0, hence i = r = k − 2, Theorem 2.8 has the following interpretation:

Theorem 0.4. We fix integers k ≥ 3, r = k − 2 and g = (k − 1)^{2}. The following locus

D_{g:k} := {[C, η] ∈ R_{g} : ∃L ∈ W_{k(k−2)}^{k−2}(C) such thatH^{0}(C, L ⊗ η) ≠ 0}

is a divisor on R_{g}. The class of its compactification inside \overline{R}_{g} is given by the formula

\overline{D}_{g:k} ≡ g!\frac{1! 2! ⋯ (k − 2)!}{(k − 1)! ⋯ (2k − 3)! (k^{2} − 2k − 1)}((k^{4} − 4k^{3} + 11k^{2} − 14k + 2)λ−

−\frac{k(k − 2)(k^{2} − 2k + 5)}{12}(δ′_{0} + δ0^{′′}) − \frac{(k^{2} − 2k + 3)(2k^{2} − 4k + 1)}{12}δ_{0}^{ram} − ⋯) ∈ Pic(\overline{R}_{g}).

When k = 3 and g = 4, the divisor D_{4:3} consists of Prym curves [C, η] ∈ R_{4} for

which the plane Prym-canonical model ι : C ⟶^{|K_{C}⊗η|} P^{2} has a triple point. Note that for a general [C, η] ∈ R_{4}, ι(C) is a 6-nodal sextic. We can then verify the formula

π_{∗}(\overline{D}_{4:3}) = 60(34λ − 4δ_{0} − 14δ_{1} − 18δ_{2}) = 60 · \overline{GP}_{4,3}^{1} ∈ Pic(\overline{M}_{4}),

where \overline{GP}_{4,3}^{1} ⊂ \overline{M}_{4} is the divisor of curves with a vanishing theta-null. This is consistent with the set-theoretic equality π(D_{4:3}) = GP_{4,3}^{1} which can be easily established (see Theorem 5.4).

Another case which has a simple interpretation is when b = 1, i = r − 1, and then g = (2k − 1)(k − 1), d = 2k(k − 1). Since rank(Q_{L}) = r and det(Q_{L}) = L, by duality we have that ∧^{i}Q_{L} = M_{L} ⊗ L, hence points [C, η] ∈ D_{(2k−1)(k−1):k} can be described purely in terms of multiplication maps of sections of line bundles on C:

Theorem 0.5. We fix integers k ≥ 2 and g = (2k − 1)(k − 1). The following locus

D_{g:k} = {[C, η] ∈ R_{g} : ∃L ∈ W_{2k(k−1)}^{2k−2}(C) withH^{0}(L)⊗H^{0}(L⊗η) → H^{0}(L^{⊗2}⊗η) not bijective}

is a divisor on R_{g}. The class of its compactification inside \overline{R}_{g} equals

\overline{D}_{g:k} ≡ g!\frac{1! 2! ⋯ (k − 1)!}{3(2k^{2} − 3k − 1)(2k − 1)! (2k)! ⋯ (3k − 2)!}(6(8k^{5}−36k^{4}+78k^{3}−95k^{2}+49k−6)λ−

−(8k^{5} − 36k^{4} + 70k^{3} − 71k^{2} + 29k − 2)(δ′_{0} + δ0^{′′})−

−\frac{1}{2}(32k^{5} − 144k^{4} + 262k^{3} − 245k^{2} + 107k − 14)δ_{0}^{ram} − ⋯).

The second class of (virtual) divisors is provided by Koszul divisors on \overline{R}_{g}. For a pair (C, L) consisting of a curve [C] ∈ M_{g} and a line bundle L ∈ Pic(C), we denote by K_{i,j}(C, L) its (i, j)-th Koszul cohomology group, cf. [L]. For a point [C, η] ∈ R_{g} we set L := K_{C} ⊗ η and we stratify R_{g} using the syzygies of the Prym-canonical curve

C →^{|L|} P^{g−2}. We define the stratum

U_{g,i} := {[C, η] ∈ R_{g} : K_{i,2}(C, K_{C} ⊗ η) ≠ ∅},

that is, U_{g,i} consists of those Prym curves [C, η] ∈ R_{g} for which the Prym-canonical

model C ⟶^{|L|} P^{g−2} fails to satisfy the Green-Lazarsfeld property (N_{i}) in the sense of [GL], [L].

Theorem 0.6. There exist two vector bundles G_{i,2} and H_{i,2} of the same rank defined over a partial compactification R̃_{2i+6} of the stack R_{2i+6}, together with a morphism ϕ : H_{i,2} → G_{i,2} such that

U_{2i+6,i} := {[C, η] ∈ R̃_{2i+6} : K_{i,2}(C, K_{C} ⊗ η) ≠ 0}

is the degeneracy locus of the map ϕ. The virtual class of [\overline{U}_{2i+6,i}] is given by the formula:

[\overline{U}_{2i+6,i}]^{virt} = c_{1}(G_{i,2} − H_{i,2}) = \binom{2i + 2}{i}(\frac{3(2i + 7)}{i + 3}λ − \frac{3}{2}δ_{0}^{ram} − (δ′_{0} + α δ0^{′′}) − ⋯),

where the constant α satisfies α ≥ 1.

The compactification R̃_{g} has the property that if R̃_{g} ⊂ \overline{R}_{g} denotes its coarse moduli space, then codim(π^{−1}(M_{g} ∪ Δ_{0}) − R̃_{g}) ≥ 2. In particular Theorem 0.6 precisely determines the coefficient of λ, δ′_{0}, δ0^{′′} and δ_{0}^{ram} in the expansion of [\overline{U}_{2i+6,i}]^{virt}. We also show that when g < 2i+6 then K_{i,2}(C, K_{C} ⊗η) ≠ ∅ for any [C, η] ∈ R_{g}. By analogy with the case of canonical curves and the classical M. Green Conjecture on syzygies of canonical curves (see [Vo]), we conjecture that the morphism of vector bundles ϕ : G_{i,2} → H_{i,2} over R̃_{2i+6} is generically non-degenerate:

Conjecture 0.7. (Prym-Green Conjecture) For a generic point [C, η] ∈ R_{g} and g ≥ 2i + 6,

we have that K_{i,2}(C, K_{C} ⊗η) = 0. Equivalently, the Prym-canonical curve C ↪^{|K_{C}⊗η|} P^{g−2} satisfies the Green-Lazarsfeld property (N_{i}) whenever g ≥ 2i + 6. For g = 2i + 6, the locus U_{2i+6,i} is an effective divisor on R_{2i+6}.

Proposition 3.1 shows that, if true, Conjecture 0.7 is sharp. In [F4] we verify the Prym-Green Conjecture for g = 2i + 6 with 0 ≤ i ≤ 5. In particular, this together with Theorem 0.6 proves that \overline{R}_{g} is of general type for g = 14, 16.

The strata U_{g,i} have been considered before for i = 0, 1, in connection with the Prym-Torelli problem. Unlike the classical Torelli problem, the Prym-Torelli problem is a subtle question: Donagi’s tetragonal construction shows that P_{g} fails to be injective over points [C, η] ∈ π^{−1}(M_{g,4}^{1}) where the curve C is tetragonal (cf. [D2]). The locus U_{g,0} consists of those points [C, η] ∈ R_{g} where the differential

(dP_{g})_{[C,η]} : H^{0}(C, K_{C}^{⊗2})^{∨} → (Sym^{2}H^{0}(C, K_{C} ⊗ η))^{∨}

is not injective and thus the infinitesimal Prym-Torelli theorem fails. It is known that (dP_{g})_{[C,η]} is generically injective for g ≥ 6 (cf. [B], or [De] Corollaire 2.3), that is, U_{g,0} is a proper subvariety of R_{g}. In particular, for g = 6 the locus U_{6,0} is a divisor of R_{6}, which gives another proof of Conjecture 0.7 for i = 0.

Debarre proved that U_{g,1} is a proper subvariety of R_{g} for g ≥ 9 (cf. [De] Théoreme 2.2). This immediately implies that for g ≥ 9 the Prym map P_{g} is generically injective, hence the Prym-Torelli theorem holds generically. Debarre’s proof unfortunately does not cover the interesting case g = 8, when U_{8,1} ⊂ R_{8} is an effective divisor (cf. [F4]).

The proof of Theorem 0.1 is finished in Section 4, using in an essential way results from [F3]: We set g′ := 1 + \frac{g−1}{g} \binom{2g}{g−1} and then we consider the rational map which associates to a curve one of its Brill-Noether loci

ϕ : \overline{M}_{2g−1} − − > \overline{M}_{1+ \frac{g−1}{g} \binom{2g}{g−1}}, ϕ[Y] := W_{g+1}^{1}(Y),

where W_{g+1}^{1}(Y) := \{L ∈ Pic^{g+1}(Y) : h^{0}(Y, L) ≥ 2\}. If χ : \overline{R}_{g} → \overline{M}_{2g−1} is the map given by χ([C, η]) := [C̃], where f : C̃ → C is the étale double cover with the property that f_{∗}OC̃ = O_{C} ⊕ η, then using [F3] we compute the slope of myriads of effective divisors of type χ^{∗}ϕ^{∗}(A), where A ∈ Ample(\overline{M}_{g′}). This proves Theorem 0.1 for even genus g = 2i + 6 ≥ 18.

We mention in passing as an immediate application of Proposition 1.9, a different proof of the statement that \overline{R}_{g} has good rationality properties for low g (see again the Introduction for the history of this problem). Our proof is quite simple and uses only numerical properties of Lefschetz pencils of curves on K3 surfaces:

Theorem 0.8. For all g ≤ 7, the Kodaira dimension of \overline{R}_{g} is −∞.

We close by summarizing the structure of the paper. In Section 1 we introduce the stack \overline{R}_{g} of Prym curves and determine the Chern classes of certain tautological vector bundles. In Section 2 we carry out the enumerative study of the divisors \overline{D}_{g:k} while in Section 3 we study Koszul divisors on \overline{R}_{g} in connection with the Prym-Green Conjecture. The proof of Theorem 0.1 is completed in Section 4 while Section 5 is concerned with the enumerative geometry of \overline{R}_{g} for g ≤ 5. In Section 6 we describe the behaviour of singularities of pluricanonical forms of \overline{R}_{g}. There is a significant overlap between some of the results of this paper and those of [Be]. Among the things we use from [Be] we mention the description of the branch locus of π and the fact that \overline{R}_{g} is isomorphic to the coarse moduli space of \overline{M}_{g}(Bℤ_{2}) (see Section 1). However, some of the results in [Be] are not correct, in particular the statement in [Be] Chapter 3 on singularities of \overline{R}_{g}. Hence we carried out a detailed study of singularities of \overline{R}_{g} in Section 6 of our paper.

1. The stack of Prym curves

In this section we review a few facts about compactifications of R_{g}. As a matter of terminology, if M is a Deligne-Mumford stack, we denote by M its coarse moduli space (This is contrary to the convention set in [ACV] but it makes sense, at least from a historical point of view). All the Picard groups of stacks or schemes we are going to consider are with rational coefficients.

We recall that π : R_{g} → M_{g} is the (2^{2g} − 1)-sheeted cover which forgets the point of order 2 and we denote by \overline{R}_{g} the normalization of \overline{M}_{g} in the function field of R_{g}. By definition, \overline{R}_{g} is a normal variety and π extends to a finite ramified covering π : \overline{R}_{g} → \overline{M}_{g}. The local behaviour of this branched cover has been studied in the thesis of M. Bernstein [Be] as well as in the paper [BCF]. In particular, the scheme \overline{R}_{g} has two distinct modular incarnations which we now recall. If X is a nodal curve, a smooth rational component E ⊂ X is said to be exceptional if #(E ∩ \overline{X − E}) = 2. The curve X is said to be quasi-stable if any two exceptional components of X are disjoint. Thus a quasi-stable curve is obtained from a stable curve by blowing-up each node at most once. We denote by [st(X)] ∈ \overline{M}_{g} the stable model of X. We have the following definition (cf. [BCF]):

Definition 1.1. A Prym curve of genus g consists of a triple (X, η, β), where X is a genus g quasi-stable curve, η ∈ Pic^{0}(X) is a line bundle of degree 0 such that η_{E} = O_{E}(1) for every exceptional component E ⊂ X, and β : η^{⊗2} → O_{X} is a sheaf homomorphism which is generically non-zero along each non-exceptional component of X.

A family of Prym curves over a base scheme S consists of a triple (X →^{f} S, η, β), where f : X → S is a flat family of quasi-stable curves, η ∈ Pic(X) is a line bundle and β : η^{⊗2} → O_{X} is a sheaf homomorphism, such that for every point s ∈ S the restriction (X_{s}, η_{X_{s}}, β_{X_{s}} : η_{X_{s}}^{⊗2} → O_{X_{s}}) is a Prym curve.

We denote by \overline{R}_{g} the non-singular Deligne-Mumford stack of Prym curves of genus g. The main result of [BCF] is that the coarse moduli space of \overline{R}_{g} is isomorphic to the normalization of \overline{M}_{g} in the function field of R_{g}. On the other hand, it is proved in [Be] that \overline{R}_{g} is also isomorphic to the coarse moduli space of the Deligne-Mumford stack \overline{M}_{g}(Bℤ_{2}) of ℤ_{2}-admissible double covers introduced in [B] and later in [ACV]. For intersection theory calculations the language of Prym curves is better suited than that of admissible covers. In particular, the existence of a degree 0 line bundle η over the universal Prym curve will be often used to compute the Chern classes of various tautological vector bundles defined over \overline{R}_{g}. Throughout this paper we use the isomorphism between rational Picard groups ϵ^{∗} : Pic(\overline{R}_{g}) → Pic(\overline{R}_{g}) induced by the map ϵ : \overline{R}_{g} → \overline{R}_{g} from the stack to its coarse moduli space.

Remark 1.2. If (X, η, β) is a Prym curve with exceptional components E_{1}, …, E_{r} and \{p_{i}, q_{i}\} = E_{i} ∩ \overline{X − E_{i}} for i = 1, …, r, then obviously β_{E_{i}} = 0. Moreover, if X̃ := \overline{X − ⋃_{i=1}^{r} E_{i}} (viewed as a subcurve of X), then we have an isomorphism of sheaves

(1)  η⊗2 ; X̃ →^{∼} OX̃(−p_{1} − q_{1} − ⋯ − p_{r} − q_{r}).

It is straightforward to describe all Prym curves [X, η, β] ∈ \overline{R}_{g} whose stable model has a prescribed topological type. We do this when st(X) is a 1-nodal curve and we determine in the process the boundary components of \overline{R}_{g} − R_{g}.

Example 1.3. (Curves of compact type) If st(X) = C ∪ D is a union of two smooth curves C and D of genus i and g − i respectively meeting transversally at a point, we use (1) to note that X = C ∪ D (that is, X has no exceptional components). The line bundle η on X is determined by the choice of two line bundles η_{C} ∈ Pic^{0}(C) and η_{D} ∈ Pic^{0}(D) satisfying η_{C}^{⊗2} = O_{C} and η_{D}^{⊗2} = O_{D} respectively. This shows that for 1 ≤ i ≤ [g/2] the pull-back under π of the boundary divisor Δ_{i} ⊂ \overline{M}_{g} splits into three irreducible components

π^{∗}(Δ_{i}) = Δ_{i} + Δ_{g−i} + Δ_{i:g−i},

where the generic point of Δ_{i} ⊂ \overline{R}_{g} is of the form [C ∪ D, η_{C} ≠ O_{C}, η_{D} = O_{D}], the generic point of Δ_{g−i} is of the form [C ∪ D, η_{C} = O_{C}, η_{D} ≠ O_{D}]), and finally Δ_{i:g−i} is the closure of the locus of points [C ∪ D, η_{C} ≠ O_{C}, η_{D} ≠ O_{D}] (see also [Be] pg. 9).

Example 1.4. (Irreducible one-nodal curves) If st(X) = C_{yq} := C/y ∼ q, where [C, y, q] ∈ M_{g−1,2}, then there are two possibilities, depending on whether X has an exceptional component or not. Suppose first that X = C′ and η ∈ Pic^{0}(X). If ν : C → X is the normalization map, then there is an exact sequence

1 ⟶ ℂ^{∗} ⟶ Pic^{0}(X) ⟶^{ν^{∗}} Pic^{0}(C) ⟶ 0.

Thus η is determined by a (non-trivial) line bundle η_{C} := ν^{∗}(η) ∈ Pic^{0}(C) satisfying η_{C}^{⊗2} = O_{C} together with an identification of the fibres η_{C}(y) and η_{C}(q). If η_{C} = O_{C}, then there is a unique way to identify the fibres η_{C}(y) and η_{C}(q) such that η ≠ O_{X}, and this corresponds to the classical Wirtinger cover of X. We denote by Δ0^{′′} = Δ_{0}^{wir} the closure in \overline{R}_{g} of the locus of Wirtinger covers. If η_{C} ≠ O_{C}, then for each such choice of η_{C} ∈ Pic^{0}(C)[2] there are 2 ways to glue η_{C}(y) and η_{C}(q). This provides another 2 × (2^{2g−2} − 1) Prym curves having C′ as their stable model. We set Δ′_{0} ⊂ \overline{R}_{g} to be the closure of the locus of Prym curves with η_{C} ≠ O_{C}.

We now treat the case when X = C ∪_{{y,q}} E, with E being an exceptional component. Then η_{E} = O_{E}(1) and η_{C}^{⊗2} = O_{C}(−y−q). The analysis carried out in [BCF] Proposition 12, shows that π is simply ramified at each of these 2^{2g−2} Prym curves in π^{−1}([C′]). We denote by Δ_{0}^{ram} ⊂ \overline{R}_{g} the closure of the locus of Prym curves [C ∪_{\{y,q\}} E, η, β] and then Δ_{0}^{ram} is the ramification divisor of π. Moreover one has the relation,

π^{∗}(Δ_{0}) = Δ′_{0} + Δ0^{′′} + 2Δ_{0}^{ram}.

It is easy to establish a dictionary between Prym curves and Beauville admissible covers. We explain how to do this in codimension 1 in \overline{R}_{g} (see also [D2] Example 1.9). The general point of Δ′_{0} corresponds to an étale double cover [C̃ →^{f} C] ∈ R_{g−1} induced by η_{C}. We denote by y_{i}, q_{i}(i = 1, 2) the points lying in f^{−1}(y) and f^{−1}(q) respectively. Then

\overline{M}_{2g−1} ∋ \frac{C̃}{y_{1} ∼ q_{1}, y_{2} ∼ q_{2}} ⟶ \frac{C}{y ∼ q} ∈ \overline{M}_{g}

is a admissible double cover, defined up to a sign. This ambiguity is then resolved in the choice of an element in Ker{ν^{∗} : Pic^{0}(C_{yq})[2] → Pic^{0}(C)[2]}.

If [C/y ∼ q, η, β] is a general point of Δ0^{′′}, then we take identical copies [C_{1}, y_{1}, q_{1}] and [C_{2}, y_{2}, q_{2}] of [C, y, q] ∈ M_{g−1,2}. The Wirtinger cover is obtained by taking

\overline{M}_{2g−1} ∋ \frac{C_{1} ∪ C_{2}}{y_{1} ∼ q_{2}, y_{2} ∼ q_{1}} ⟶ \frac{C}{y ∼ q} ∈ \overline{M}_{g}.

If [C ∪_{\{y,q\}} E, η, β] ∈ Δ_{0}^{ram}, then η_{C} ∈ \sqrt{O_{C}(−y − q)} induces a 2 : 1 cover C̃ →^{f} C

branched over y and q. We set {ỹ} := f^{−1}(y), {q̃} := f^{−1}(q). The Beauville cover is

\overline{M}_{2g−1} ∋ \frac{C̃}{ỹ ∼ q̃} ⟶ \frac{C}{y ∼ q} ∈ \overline{M}_{g}.

As usual, one denotes by δ′_{0}, δ0^{′′}, δ_{0}^{ram}, δ_{i}, δ_{g−i}, δ_{i:g−i} ∈ Pic(\overline{R}_{g}) the stacky divisor classes corresponding to the boundary divisors of \overline{R}_{g}. We also set λ := π^{∗}(λ) ∈ Pic(\overline{R}_{g}). Next we determine the canonical class K_{\overline{R}_{g}}:

Theorem 1.5. One has the following formula in Pic(\overline{R}_{g}):

K_{\overline{R}_{g}} = 13λ − 2(δ0^{′} + δ0^{′′}) − 3δ_{0}^{ram} − 2 ∑_{i=1}^{[g/2]}(δ_{i} + δ_{g−i} + δ_{i:g−i}) − (δ_{1} + δ_{g−1} + δ_{1:g−1}).

Proof. We use that K_{\overline{M}_{g}} ≡ 13λ−2δ_{0}−3δ_{1}−2δ_{2}−⋯−2δ_{[g/2]} (cf. [HM]), together with the Hurwitz formula for the cover π : \overline{R}_{g} → \overline{M}_{g}. We find that K_{\overline{R}_{g}} = π^{∗}(K_{\overline{M}_{g}}) + δ_{0}^{ram}. □

Using this formula as well as the Appendix, we conclude that in order to prove that \overline{R}_{g} is of general type for a certain g, it suffices to exhibit a single effective divisor

D ≡ aλ − b′_{0}δ′_{0} − b0^{′′}δ0^{′′} − b_{0}^{ram}δ_{0}^{ram} − ∑_{i=1}^{[g/2]}(b_{i}δ_{i} + b_{g−i}δ_{g−i} + b_{i:g−i}δ_{i:g−i}) ∈ Eff(\overline{R}_{g}),

satisfying the following inequalities:

(2)  max\{\frac{a}{b′_{0}} , \frac{a}{b0^{′′}} \} < \frac{13}{2}, max\{\frac{a}{b_{0}^{ram}} , \frac{a}{b_{1}} , \frac{a}{b_{g−1}} , \frac{a}{b_{1:g−1}} \} < \frac{13}{3}

and

max_{i≥1}\{\frac{a}{b_{i}} , \frac{a}{b_{g−i}} , \frac{a}{b_{i:g−i}} \} < \frac{13}{2}.

1.1. The universal Prym curve. We start by introducing the partial compactification M̃_{g} := M_{g} ∪ Δ̃_{0} of M_{g}, obtaining by adding to M_{g} the locus Δ̃_{0} ⊂ \overline{M}_{g} of one-nodal irreducible curves [C_{yq} := C/y ∼ q], where [C, y, q] ∈ M_{g−1,2}. Let p : M̃_{g,1} → M̃_{g} denote the universal curve. We denote R̃_{g} := π^{−1}(M̃_{g}) ⊂ \overline{R}_{g} and note that the boundary divisors Δ̃′_{0} := Δ′_{0} ∩ R̃_{g}, Δ̃0^{′′} := Δ0^{′′} ∩ R̃_{g} and Δ̃_{0}^{ram} := Δ_{0}^{ram} ∩ R̃_{g} become disjoint inside R̃_{g}. Finally, we set Z := R̃_{g} ×_{M̃_{g}} M̃_{g,1} and denote by p_{1} : Z → R̃_{g} the projection.

To obtain the universal family of Prym curves over R̃_{g}, we blow-up the codimension 2 locus V ⊂ Z corresponding to points

v = ([C ∪_{\{y,q\}} E, η_{C} ∈ \sqrt{O_{C}(−y − q)}], η_{E} = O_{E}(1), ν(y) = ν(q)) ∈ Δ_{0}^{ram} ×_{M̃_{g}} M̃_{g,1}

(recall that ν : C → C_{yq} denotes the normalization map). Suppose that (t_{1}, …, t_{3g−3}) are local coordinates in an étale neighbourhood of [C ∪_{{y,q}} E, η_{C}, η_{E}] ∈ R̃_{g} such that the local equation of Δ_{0}^{ram} is (t_{1} = 0). Then Z around v admits local coordinates (x, y, t_{1}, …, t_{3g−3}) satisfying the equation xy = t_{1}^{2}. In particular, Z is singular along V. We denote by X := Bl_{V} (Z) and by f : X → R̃_{g} the induced family of Prym curves. Then for every [X, η, β] ∈ R̃_{g} we have that f^{−1}([X, η, β]) = X.

On X there exists a Prym line bundle P ∈ Pic(X) as well as a morphism of O_{X}- modules B : P^{⊗2} → O_{X} with the property that P_{|f^{−1}([X,η,β])} = η and B_{|f^{−1}([X,η,β])} = β : η^{⊗2} → O_{X}, for all points [X, η, β] ∈ R̃_{g} (see e.g. [C], the same argument carries over from the spin to the Prym moduli space).

We set E′_{0}, E0^{′′} and E_{0}^{ram} ⊂ X to be the proper transforms of the boundary divisors p_{1}^{−1}(Δ̃′_{0}), p_{1}^{−1}(Δ̃0^{′′}) and p_{1}^{−1}(Δ̃_{0}^{ram}) respectively. Finally, we define E_{0} to be the exceptional divisor of the blow-up map X → Z.

We recall that g : Y → S is a family of nodal curves and L, M are line bundles on Y, then ⟨L, M⟩ ∈ Pic(S) denotes the bilinear Deligne pairing of L and M.

Proposition 1.6. If f : X → R̃_{g} is the universal Prym curve and P ∈ Pic(X) is the corresponding Prym bundle, then one has the following relations in Pic(R̃_{g}):

(i) ⟨ω_{f}, P⟩ = 0.
(ii) ⟨O_{X} (E_{0}), O_{X} (E_{0})⟩ = −2δ_{0}^{ram}.
(iii) ⟨O_{X} (P), O_{X} (P)⟩ = −δ_{0}^{ram}/2.

Proof. The sheaf homomorphism B : P^{⊗2} → O_{X} vanishes (with order 1) precisely along the exceptional divisor E_{0}, hence [E_{0}] = −2c_{1}(P). Furthermore, we have the relations f^{∗}(Δ_{0}^{ram}) = E_{0}^{ram} + E_{0} and f_{∗}([E_{0}^{ram}] · [E_{0}]) = 2δ_{0}^{ram} (In the fibre f^{−1}([C ∪_{{y,q}} E, η_{C}]) the divisors E_{0} and E_{0}^{ram} meet over two points, corresponding to whether the marked points equals y or q. Now (ii) and (iii) follow simply from the push-pull formula. For (i), it is enough to show that ω_{f|E_{0}} is the trivial bundle. This follows because for any point [X, η, β] ∈ R̃_{g} we have that ω_{X} ⊗ O_{E} = 0, for any exceptional component E ⊂ X. □

We now fix i ≥ 1 and set N_{i} := f_{∗}(ω_{f}^{⊗i} ⊗ P^{⊗i}). Since R^{1}f_{∗}(ω_{f}^{⊗i} ⊗ P^{⊗i}) = 0, Grauert’s theorem implies that N_{i} is a vector bundle over R̃_{g} of rank (g − 1)(2i − 1).

Proposition 1.7. For each integer i ≥ 1 the following formula in Pic(R̃_{g}) holds:

c_{1}(N_{i}) = \binom{i}{2}(12λ − δ′_{0} − δ0^{′′} − 2δ_{0}^{ram}) + λ − \frac{i^{2}}{4} δ_{0}^{ram}.

Proof. We apply Grothendieck-Riemann-Roch to the universal Prym curve f : X → R̃_{g}:

c_{1}(N_{i}) = f_{∗}[(1 + ic_{1}(ω_{f} ⊗ P) + \frac{i^{2}c_{1}^{2}(ω_{f} ⊗ P)}{2})(1 − \frac{c_{1}(ω_{f})}{2} + \frac{c_{1}^{2}(ω_{f}) + [Sing(f)]}{12})]_{2},

and then use Proposition 1.6 and Mumford’s formula (κ_{1})_{R̃_{g}} = 12λ−δ′_{0} −δ0^{′′} −2δ_{0}^{ram}. □

1.2. Inequalities between coefficients of divisors on \overline{R}_{g}. We use pencils of curves on K3 surfaces to establish certain inequalities between the coefficients of effective divisors on \overline{R}_{g}. Using K3 surfaces we construct pencils that fill up the boundary divisors Δ_{i}, Δ_{g−i} and Δ_{i:g−i} for 1 ≤ i ≤ [g/2] when g ≤ 23. The use of such pencils in the context of \overline{M}_{g} has already been demonstrated in [FP].

We start with a Lefschetz pencil B ⊂ \overline{M}_{i} of curves of genus i lying on a fixed K3 surface S. The pencil B is induced by a family f : Bl_{i^{2}}(S) → P^{1} which has i^{2} sections corresponding to the base points and we choose one such section σ. Using B, for each g ≥ i + 1 we create a genus g pencil B_{i} ⊂ \overline{M}_{g} of stable curves, by gluing a fixed curve [C_{2}, p] ∈ M_{g−i,1} along the section σ to each member of the pencil B. Then we have the following formulas on \overline{M}_{g} (cf. [FP] Lemma 2.4):

B_{i} · λ = i + 1, B_{i} · δ_{0} = 6i + 18, B_{i} · δ_{i} = −1 and B_{i} · δ_{j} = 0 for j ≠ i.

We fix 1 ≤ i ≤ [g/2] and lift B_{i} in three different ways to pencils in \overline{R}_{g}. First we choose a non-trivial line bundle η_{2} ∈ Pic^{0}(C_{2})[2]. Let us denote by A_{g−i} ⊂ Δ_{g−i} ⊂ \overline{R}_{g} the pencil of Prym curves [C_{2} ∪_{σ(λ)} f^{−1}(λ), η_{C_{2}} = η_{2}, η_{f^{−1}(λ)} = O_{f^{−1}(λ)}], with λ ∈ P^{1}.

Next, we denote by A_{i} ⊂ Δ_{i} ⊂ \overline{R}_{g} the pencil consisting of Prym curves
[C_{2} ∪_{σ(λ)} f^{−1}(λ), η_{C_{2}} = O_{C_{2}}, η_{f^{−1}(λ)} ∈ \overline{Pic}^{0}(f^{−1}(λ))[2]], where λ ∈ P^{1}.

Clearly π(A_{i}) = B_{i} and deg(A_{i}/B_{i}) = (2^{2i} − 1). Finally, A_{i:g−i} ⊂ Δ_{i:g−i} ⊂ \overline{R}_{g} denotes the pencil of Prym curves [C_{2} ∪ f^{−1}(λ), η_{C_{2}} = η_{2}, η_{f^{−1}(λ)} ∈ \overline{Pic}^{0}(f^{−1}(λ))[2]]. Again, we have that deg(A_{i:g−i}/B_{i}) = 2^{2i} − 1.

Lemma 1.8. If A_{i}, A_{g−i} and A_{i:g−i} are pencils defined above, we have the following relations:

• A_{g−i}·λ = i+1, A_{g−i}·δ′_{0} = 6i+18, A_{g−i}·δ_{i} = A_{g−i}·δ_{0}^{ram} = 0, andA_{g−i}·δ_{g−i} = −1.

• A_{i} · λ = (i + 1)(2^{2i} − 1), A_{i} · δ′_{0} = (2^{2i−1} − 2)(6i + 18), A_{i} · δ0^{′′} = 6i + 18,

A_{i} · δ_{0}^{ram} = 2^{2i−2}(6i + 18) andA_{i} · δ_{i} = −(2^{2i} − 1).

• A_{i:g−i} · λ = (i + 1)(2^{2i} − 1), A_{i:g−i} · δ′_{0} = (2^{2i−1} − 1)(6i + 18),

A_{i:g−i} · δ_{0}^{ram} = 2^{2i−2}(6i + 18), A_{i:g−i} · δ0^{′′} = 0 and A_{i:g−i} · δ_{i:g−i} = −(2^{2i} − 1).

Note that all these intersections are computed on \overline{R}_{g}. The intersection numbers of A_{i}, A_{g−i} and A_{i:g−i} with the generators of Pic(\overline{R}_{g}) not explicitly mentioned in Lemma 1.8 are all equal to 0.

Proof. We treat in detail only the case of A_{i} the other cases being similar. Using [FP] we find that (A_{i} · λ)_{\overline{R}_{g}} = (π_{∗}(A_{i}) · λ)_{\overline{M}_{g}} = (2^{2i} − 1)(B_{i} · λ)_{\overline{M}_{g}}. Furthermore, since A_{i} ∩ Δ_{g−i} = A_{i} ∩ Δ_{i:g−i} = ∅, we can write the formulas

(A_{i} · δ_{i})_{\overline{R}_{g}} = (A_{i} · π^{∗}(δ_{i}))_{\overline{R}_{g}} = (2^{2i} − 1)(B_{i} · δ_{i})_{\overline{M}_{g}}.

Clearly (A_{i} ·δ0^{′′})_{\overline{R}_{g}} = (B_{i} ·δ_{0})_{\overline{M}_{g}} = 6i+18, whereas the intersection A_{i} ·δ′_{0} corresponds to choosing an element in Pic^{0}(f^{−1}(λ))[2], where f^{−1}(λ) is a singular member of B. There are 2(2^{2i−2} − 1)(6i + 18) such choices. □

Proposition 1.9. Let D ≡ aλ−b′_{0}δ′_{0}−b0^{′′}δ0^{′′}−b_{0}^{ram}δ_{0}^{ram}−∑_{i=1}^{[g/2]}(b_{i}δ_{i}+b_{g−i}δ_{g−i}+b_{i:g−i}δ_{i:g−i}) ∈

Pic(\overline{R}_{g}) be the closure in \overline{R}_{g} of an effective divisor in R_{g}. Then if 1 ≤ i ≤ min\{[g/2], 11\}, we have the following inequalities:

(1) a(i + 1) − b′_{0}(6i + 18) + b_{g−i} ≥ 0.

(2) a(i + 1) − b_{0}^{ram}(6i + 18)\frac{2^{2i−2}}{2^{2i}−1} − b′_{0}(6i + 18)\frac{2^{2i−1}−1}{2^{2i}−1} + b_{i:g−i} ≥ 0.

(3) a(i + 1) − b_{0}^{ram}(6i + 18)\frac{2^{2i−2}}{2^{2i}−1} − b′_{0} (6i + 18)\frac{2^{2i−1}−2}{2^{2i}−1} − b0^{′′}(6i + 18)\frac{1}{2^{2i}−1} + b_{i} ≥ 0.

Proof. We use that that in this range the pencils A_{i}, A_{g−i} and A_{i:g−i} fill-up the boundary divisors Δ_{i}, Δ_{g−i} and Δ_{i:g−i} respectively, hence A_{i} · D, A_{g−i} · D, A_{i:g−i} · D ≥ 0. □

Proof of Theorem 0.8. We lift the Lefschetz pencil B ⊂ \overline{M}_{g} corresponding to a fixed K3 surface, to a pencil B̃ ⊂ \overline{R}_{g} of Prym curves by taking Prym curves B̃ := \{[C_{λ}, η_{C_{λ}}] ∈ \overline{R}_{g} : [C_{λ}] ∈ B, η_{C_{λ}} ∈ \overline{Pic}^{0}(C_{λ})[2]\}. We have the following formulas

B̃·λ = (2^{2g}−1)(g+1), B̃·δ′_{0} = (2^{2g−1}−2)(6g+18), B̃·δ0^{′′} = 6g+18, B̃·δ_{0}^{ram} = 2^{2g−2}(6g+18).

Furthermore, B̃ is disjoint from all the remaining boundary classes of \overline{R}_{g}. One now verifies that B̃ · K_{\overline{R}_{g}} < 0 precisely when g ≤ 7. Since B̃ is a covering curve for \overline{R}_{g} in the range g ≤ 11, g ≠ 10, we find that κ(\overline{R}_{g}) = −∞. □

2. Theta divisors for vector bundles and geometric loci in \overline{R}_{g}

We present a general method of constructing geometric divisors on \overline{R}_{g}. For a fixed point [C, η] ∈ R_{g} we shall study the relative position of η ∈ Pic^{0}(C)[2] with respect to certain pluri-theta divisors on Pic^{0}(C).

We start by fixing a smooth curve C. If E ∈ U_{C}(r, d) is a semistable vector bundle on C of integer slope μ(E) := d/r ∈ ℤ, then following Raynaud [R], we introduce the determinantal cycle

Θ_{E} := {η ∈ Pic^{g−μ−1}(C) : H^{0}(C, E ⊗ η) ≠ 0}.

Either Θ_{E} = Pic^{g−μ−1}(C), or else, Θ_{E} is a divisor on Pic^{g−μ−1}(C) and then Θ_{E} ≡ r · θ. In the latter case we say that Θ_{E} is the theta divisor of the vector bundle E. Clearly, Θ_{E} is a divisor if and only if H^{0}(C, E ⊗ η) = 0, for a general bundle η ∈ Pic^{g−μ−1}(C).

Let us now fix a globally generated line bundle L ∈ Pic^{d}(C) such that h^{0}(C, L) = r + 1. The Lazarsfeld vector bundle M_{L} of L is defined using the exact sequence on C

0 ⟶ M_{L} ⟶ H^{0}(C, L) ⊗ O_{C} ⟶ L ⟶ 0

(see also [GL], [L], [Vo], [F1], [FMP] for many applications of these bundles). It is customary to denote Q_{L} := M_{L}^{∨}, hence μ(Q_{L}) = d/r. When L = K_{C}, one writes Q_{C} := Q_{K_{C}}. The vector bundles Q_{L} (and all its exterior powers) are semistable under mild genericity assumptions on C (see [L] or [F1] Proposition 2.1). In the case μ(∧^{i}Q_{L}) = g−1, when we expect Θ_{∧^{i}Q_{L}} to be a divisor on Pic^{0}(C), we may ask whether for a given point [C, η] ∈ R_{g} the condition η ∈ Θ_{∧^{i}Q_{L}} is satisfied or not. Throughout this section we denote by 𝔊_{d}^{r} → M_{g} the Deligne-Mumford stack parameterizing pairs [C, l], where [C] ∈ M_{g} and l = (L, V) ∈ G_{d}^{r}(C) is a linear series of type 𝔤_{d}^{r}.

We fix integers k ≥ 2 and b ≥ 0. We set integers i := kb + k − b − 2,

r := kb + k − 2, g := k(kb + k − b − 2) + 1 = ik + 1 and d := k(kb + k − 2).

Since ρ(g, r, d) = 0, a general curve [C] ∈ M_{g} carries a finite number of (obviously complete) linear series l ∈ G_{d}^{r}(C). We denote this number by

N := g!\frac{1! 2! ⋯ r!}{(k − 1)! ⋯ (k − 1 + r)!} = deg(𝔊_{d}^{r}/M_{g}).

We also note that we can write g = (r + 1)(k − 1) and d = rk, and moreover, each line bundle L ∈ W_{d}^{r}(C) satisfies h^{1}(C, L) = k − 1. Furthermore, we compute μ(∧^{i}Q_{L}) = ik = g − 1 and then we introduce the following virtual divisor on R_{g}:

D_{g:k} := {[C, η] ∈ R_{g} : ∃L ∈ W_{d}^{r}(C) such that h^{0}(C, ∧^{i}Q_{L} ⊗ η) ≥ 1}.

From the definition it follows that D_{g:k} is either pure of codimension 1 in R_{g}, or else D_{g:k} = R_{g}. We shall prove that the second possibility does not occur.

For [C, η] ∈ R_{g} and L ∈ W_{d}^{r}(C) one has the following exact sequence on C

0 ⟶ ∧^{i}M_{L} ⊗ K_{C} ⊗ η ⟶ ∧^{i}H^{0}(C, L) ⊗ K_{C} ⊗ η ⟶ ∧^{i−1}M_{L} ⊗ L ⊗ K_{C} ⊗ η ⟶ 0,

from which, using Serre duality, one derives the following equivalences:

[C, η] ∈ D_{g:k} ⇔ h^{1}(C, ∧^{i}M_{L} ⊗ K_{C} ⊗ η) ≥ 1 ⇔
(3) ∧^{i}H^{0}(C, L)⊗H^{0}(C, K_{C} ⊗η) → H^{0}(C, ∧^{i−1}M_{L}⊗L⊗K_{C} ⊗η) is not an isomorphism .

Note that obviously rank(∧^{i}H^{0}(C, L) ⊗ H^{0}(C, K_{C} ⊗ η)) = \binom{r+1}{i}(g − 1), while

h^{0}(C, ∧^{i−1}M_{L} ⊗ L ⊗ K_{C} ⊗ η) = χ(C, ∧^{i−1}M_{L} ⊗ L ⊗ K_{C} ⊗ η) =

= \binom{r}{i − 1}(−k(i − 1) + d + g − 1) = \binom{r + 1}{i}(g − 1)

(use that M_{L} is a semistable vector bundle and that μ(∧^{i−1}M_{L} ⊗ L ⊗ K_{C} ⊗ η) > 2g − 1).

Remark 2.1. As pointed out in the Introduction, an important particular case is k = 2, when i = b, g = 2i + 1, r = 2i, d = 4i = 2g − 2. Since W_{2g−2}^{g−1}(C) = {K_{C}}, it follows that [C, η] ∈ D_{2i+1,2} ⇔ η ∈ Θ_{∧^{i}Q_{C}}. The main result from [FMP] states that for any [C] ∈ M_{g} the Raynaud locus Θ_{∧^{i}Q_{C}} is a divisor in Pic^{0}(C) (that is, ∧^{i}Q_{C} has a theta divisor) and we have an equality of cycles

(4)  Θ_{∧^{i}Q_{C}} = C_{i} − C_{i} ⊂ Pic^{0}(C),

where the right-hand-side denotes the i-th difference variety of C, that is, the image of the difference map

ϕ : C_{i} × C_{i} → Pic^{0}(C), ϕ(D, E) := O_{C}(D − E).

Using Lazarsfeld’s filtration argument [L] Lemma 1.4.1, one finds that for a generic choice of distinct points x_{1}, …, x_{g−2} ∈ C, there is an exact sequence

0 ⟶ ⊕_{l=1}^{g−2}O_{C}(x_{l}) ⟶ Q_{C} ⟶ K_{C} ⊗ O_{C}(−x_{1} − ⋯ − x_{g−2}) ⟶ 0,

which implies the inclusion C_{i} −C_{i} ⊂ Θ_{∧^{i}Q_{C}}. The importance of (4) is that it shows that Θ_{∧^{i}Q_{C}} is a divisor on Pic^{0}(C), that is, H^{0}(C, ∧^{i}Q_{C} ⊗ η) = 0 for a generic η ∈ Pic^{0}(C).

Theorem 2.2. For every genus g = 2i + 1 we have the following identification of cycles on R_{g}:

D_{2i+1:2} := {[C, η] ∈ R_{g} : η ∈ C_{i} − C_{i}}.

Next we prove that D_{g:k} is an actual divisor on R_{g} for any k ≥ 2 and we achieve this by specialization to the k-gonal locus M_{g,k}^{1} in M_{g}.

Theorem 2.3. Fix k ≥ 2, b ≥ 1 and g, r, d, i defined as above. Then D_{g:k} is a divisor on R_{g}. Precisely, for a generic [C, η] ∈ R_{g} we have that H^{0}(C, ∧^{i}Q_{L} ⊗ η) = 0, for every L ∈ W_{d}^{r}(C).

Proof. Since there is a unique irreducible component of 𝔊_{d}^{r}(R_{g}/M_{g}) := 𝔊_{d}^{r} ×_{M_{g}} R_{g} mapping dominantly onto R_{g}, in order to prove that D_{g:k} is a divisor it suffices to exhibit a single triple [C, L, η] ∈ 𝔊_{d}^{r}(R_{g}/M_{g}) such that (1) the Petri map

μ_{0}(C, L) : H^{0}(C, L) ⊗ H^{0}(C, K_{C} ⊗ L^{∨}) → H^{0}(C, K_{C})

is an isomorphism and (2) the torsion point η ∈ Pic^{0}(C)[2] is such that η ∉ Θ_{∧^{i}Q_{L}}.

Proposition 2.1.1 from [CM] ensures that for a generic k-gonal curve [C, A] ∈ 𝔊_{k}^{1} of genus g = (r+1)(k−1) one has that h^{0}(C, A^{⊗j}) = j +1 for 1 ≤ j ≤ r+1. In particular there is an isomorphism Sym^{j}H^{0}(C, A) ≅ H^{0}(C, A^{⊗j}). Using this and Riemann-Roch, we obtain that h^{0}(C, K_{C} ⊗ A^{⊗(−j)}) = (k − 1)(r + 1 − j) for 0 ≤ j ≤ r + 1. Thus there is a generically injective rational map 𝔊_{k}^{1} − − > 𝔊_{d}^{r} given by [C, A] ↦ [C, A^{⊗r}] (The use of such a map has been first pointed out to me in a different context by S. Keel). We claim that 𝔊_{k}^{1} maps into the ”main component” of 𝔊_{d}^{r} which maps dominantly onto \overline{M}_{g}. To prove this it suffices to check that the Petri map

μ_{0}(C, A^{⊗r}) : H^{0}(C, A^{⊗r}) ⊗ H^{0}(C, K_{C} ⊗ A^{⊗(−r)}) → H^{0}(C, K_{C})

is an isomorphism (Remember that H^{0}(C, A^{⊗r}) ≅ Sym^{r}H^{0}(C, A)). We use the base point free pencil trick to write down the exact sequence

0 ⟶ H^{0}(K_{C} ⊗ A^{⊗−(j+1)}) ⟶ H^{0}(A) ⊗ H^{0}(K_{C} ⊗ A^{⊗(−j)}) ⟶^{μ_{j}(A)} H^{0}(K_{C} ⊗ A^{⊗−(j−1)}).

One can now easily check that μ_{j}(A) is surjective for 1 ≤ j ≤ r by using the formulas h^{0}(C, K_{C} ⊗ A^{⊗(−j)}) = (k − 1)(r + 1 − j) valid for 0 ≤ j ≤ r + 1. This in turns implies that μ_{0}(C, A^{⊗r}) is surjective, hence an isomorphism.

We now check condition (2) and note that for [C, L = A^{⊗r}] ∈ 𝔊_{d}^{r}, the Lazarsfeld bundle splits as Q_{L} ≅ A^{⊕r}. In particular, ∧^{i}Q_{L} ≅ ⊕\binom{r}{i}A^{⊗i}, hence the condition

H^{0}(C, ∧^{i}Q_{L} ⊗ η) ≠ 0 is equivalent to H^{0}(C, A^{⊗i} ⊗ η) ≠ 0, that is, the translate of the theta divisor W_{g−1}(C) − A^{⊗i} ⊂ Pic^{0}(C) cannot contain all points of order 2 on Pic^{0}(C). We assume by contradiction that for any [C, A] ∈ 𝔊_{k}^{1} and any η ∈ Pic^{0}(C)[2], we have that H^{0}(C, A^{⊗i}⊗η) ≥ 1. We use that 𝔊_{k}^{1} is irreducible and specialize C to a hyperelliptic curve and choose A = 𝔤_{2}^{1} ⊗ O_{C}(x_{1} + ⋯ + x_{k−2}), with x_{1}, …, x_{k−2} ∈ C being general points. Finally we take η := O_{C}(p_{1} + ⋯ + p_{i+1} − q_{1} − ⋯ − q_{i+1}) ∈ Pic^{0}(C)[2], with p_{1}, …, p_{i+1}, q_{1}, …, q_{i+1} being distinct ramification points of the hyperelliptic 𝔤_{2}^{1}. It is now straightforward to check that H^{0}(C, A^{⊗i} ⊗ η) = 0. □

In order to compute the class [\overline{D}_{g:k}] ∈ Pic(\overline{R}_{g}) we extend the determinantal description of D_{g:k} to the boundary of \overline{R}_{g}. We start by setting some notation. We denote by M_{g}^{0} ⊂ M_{g} the open substack classifying curves [C] ∈ M_{g} such that W_{d−1}^{r}(C) = ∅ and W_{d}^{r+1}(C) = ∅. We know that codim(M_{g} − M_{g}^{0}, M_{g}) ≥ 2. We further denote by Δ_{0}^{0} ⊂ Δ_{0} ⊂ \overline{M}_{g} the locus of curves [C/y ∼ q] where [C] ∈ M_{g−1} is a curve that satisfies the Brill-Noether theorem and where y, q ∈ C are arbitrary points. Note that every Brill-Noether general curve [C] ∈ M_{g−1} satisfies

W_{d−1}^{r}(C) = ∅, W_{d}^{r+1}(C) = ∅ and dim W_{d}^{r}(C) = ρ(g − 1, r, d) = r.

We set \overline{M}_{g}^{0} := M_{g}^{0} ∪ Δ_{0}^{0} ⊂ \overline{M}_{g}. Then we consider the Deligne-Mumford stack

σ_{0} : 𝔊_{d}^{r} → \overline{M}_{g}^{0}

classifying pairs [C, L] with [C] ∈ \overline{M}_{g}^{0} and L ∈ G_{d}^{r}(C) (cf. [EH], [F2], [Kh] -note that it is essential that ρ(g, r, d) = 0. At the moment there is no known extension of this stack over the entire \overline{M}_{g}). We remark that for any curve [C] ∈ \overline{M}_{g}^{0} and L ∈ W_{d}^{r}(C) we have that h^{0}(C, L) = r + 1, that is, 𝔊_{d}^{r} parameterizes only complete linear series. Indeed, for a smooth curve [C] ∈ M_{g}^{0} we have that W_{d}^{r+1}(C) = ∅, so necessarily W_{d}^{r}(C) = G_{d}^{r}(C). For a point [C_{yq} := C/y ∼ q] ∈ Δ_{0}^{0} we have the identification

σ_{0}^{−1}[C_{yq}] = {L ∈ W_{d}^{r}(C) : h^{0}(C, L ⊗ O_{C}(−y − q)) = r},

where we note that since the normalization [C] ∈ M_{g−1} is assumed to be Brill-Noether general, any sheaf L ∈ σ_{0}^{−1}[C_{yq}] satisfies h^{0}(C, L ⊗ O_{C}(−y)) = h^{0}(C, L ⊗ O_{C}(−q)) = r and h^{0}(C, L) = r + 1. Furthermore, σ_{0} : 𝔊_{d}^{r} → \overline{M}_{g}^{0} is proper, which is to say that \overline{W} _{d}^{r}(C_{yq}) = W_{d}^{r}(C_{yq}), where the left-hand-side denotes the closure of W_{d}^{r}(C_{yq}) in the variety \overline{Pic}^{d}(C_{yq}) of torsion-free sheaves on C_{yq}. This follows because a non-locally free torsion-free sheaf in \overline{W} _{d}^{r}(C_{yq}) − W_{d}^{r}(C_{yq}) is of the form ν_{∗}(A), where A ∈ W_{d−1}^{r}(C) and ν : C → C_{yq} is the normalization map. But we know that W_{d−1}^{r}(C) = ∅, because

[C] ∈ M_{g−1} satisfies the Brill-Noether theorem. Since ρ(g, r, d) = 0, by general Brill-Noether theory, there exists a unique irreducible component of 𝔊_{d}^{r} which maps onto \overline{M}_{g}^{0}. It is certainly not the case that 𝔊_{d}^{r} is irreducible, unless k ≤ 3, when either 𝔊_{d}^{r} = M_{g} (k = 2), or 𝔊_{d}^{r} is isomorphic to a Hurwitz stack (k = 3). We denote by f_{d}^{r} : ℭ_{g,d}^{r} :=

\overline{M}_{g,1}^{0}×\overline{M}{0 \atop g} 𝔊_{d}^{r} → 𝔊_{d}^{r} the pull-back of the universal curve \overline{M}_{g,1}^{0} → \overline{M}_{g}^{0} to 𝔊_{d}^{r}. Once we have chosen a Poincaré bundle L on ℭ_{g,d}^{r} we can form the three codimension 1 tautological classes in A^{1}(𝔊_{d}^{r}): (5) 𝔞 := (f_{d}^{r})_{∗}(c_{1}(L)^{2}), 𝔟 := (f_{d}^{r})_{∗}(c_{1}(L) · c_{1}(ω_{f_{d}^{r}})), 𝔠 := (f_{d}^{r})_{∗}(c_{1}(ω_{f_{d}^{r}})^{2}) = (σ_{0})^{∗}((κ_{1})\overline{M}{0 \atop g}).

These classes depend on the choice of L and behave functorially with respect to base change, see also Remark 2.7 on the precise statement regarding the choice of L. We set \overline{R}_{g}^{0} := π^{−1}(M̃_{g}^{0}) ⊂ R̃_{g} and introduce the stack of 𝔤_{d}^{r}’s on Prym curves

σ : 𝔊_{d}^{r}(R̃_{g}^{0}/M̃_{g}^{0}) := \overline{R}_{g}^{0} ×\overline{M}{0 \atop g} 𝔊_{d}^{r} → \overline{R}_{g}^{0}.

By a slight abuse of notation we denote the boundary divisors by the same symbols, that is, Δ′_{0} := σ^{∗}(Δ′_{0}), Δ0^{′′} := σ^{∗}(Δ0^{′′}) and Δ_{0}^{ram} := σ^{∗}(Δ_{0}^{ram}). Finally, we introduce the universal curve over the stack of 𝔤_{d}^{r}’s on Prym curves:

f′ : X_{d}^{r} := X ×\overline{R}{0 \atop g} 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}) → 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}).

On X_{d}^{r} there are two tautological line bundles, the universal Prym bundle P_{d}^{r} which is the pull-back of P ∈ Pic(X) under the projection X_{d}^{r} → X, and a Poincaré bundle L ∈ Pic(X_{d}^{r}) characterized by the property L_{|f^{′−1}[X,η,β,L]} = L ∈ W_{d}^{r}(C), for each point [X, η, β, L] ∈ 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}). Note that we also have the codimension 1 classes 𝔞, 𝔟, 𝔠 ∈ A^{1}(𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0})) defined by the formulas (5).

Proposition 2.4. Let C be a curve of genus g and let L ∈ W_{d}^{r}(C) be a globally generated complete linear series. Then for any integer 0 ≤ j ≤ r and for any line bundle A ∈ Pic^{a}(C) such that a ≥ 2g + d − r + j − 1, we have that H^{1}(C, ∧^{j}M_{L} ⊗ A) = 0.

Proof. We use a filtration argument due to Lazarsfeld [L]. Having fixed L and A, we choose general points x_{1}, …, x_{r−1} ∈ C such that h^{0}(C, L ⊗ O_{C}(−x_{1} − ⋯ − x_{r−1})) = 2 and then there is an exact sequence on C

0 ⟶ L^{∨}(x_{1} + ⋯ + x_{r−1}) ⟶ M_{L} ⟶ ⊕_{l=1}^{r−1}O_{C}(−x_{l}) ⟶ 0.

Taking the j-th exterior powers and tensoring the resulting exact sequence with A, we find that in order to conclude that H^{1}(C, ∧^{i}M_{L} ⊗ A) = 0 for i ≤ r, it suffices to show that for 1 ≤ i ≤ r the following hold:

(1) H^{1}(C, A ⊗ O_{C}(−D_{j})) = 0 for each effective divisor D_{j} ∈ C_{j} with support in the set {x_{1}, …, x_{r−1}}, and

(2) H^{1}(C, A ⊗ L^{∨} ⊗ O_{C}(D_{r−j})) = 0, for any effective divisor D_{r−j} ∈ C_{r−j} with support contained in {x_{1}, …, x_{r−1}}.

Both (1) and (2) hold for degree reasons since deg(C, A ⊗ O_{C}(−D_{j})) ≥ 2g − 1 and deg(C, A ⊗ L^{∨} ⊗ O_{C}(D_{r−j}) ≥ 2g − 1 and the points x_{1}, …, x_{r−1} ∈ C are general. □

Next we use Proposition 2.4 to prove a vanishing result for Prym curves.

Proposition 2.5. For each point [X, η, β, L] ∈ 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}) and 0 ≤ a ≤ i − 1, we have that

H^{1}(X, ∧^{a}M_{L} ⊗ L^{⊗(i−a)} ⊗ ω_{X} ⊗ η) = 0.

Proof. If X is smooth, then the vanishing follows directly from Proposition 2.4. Assume now that [X, η, β] ∈ Δ′_{0} ∪ Δ0^{′′}, that is, st(X) = X and η ∈ Pic^{0}(X)[2]. As usual, we denote by ν : C → X the normalization map, and L_{C} := ν^{∗}(L) ∈ W_{d}^{r}(C) satisfies h^{0}(C, L_{C} ⊗O_{C}(−y−q)) = r, hence H^{0}(X, L) ≅ H^{0}(C, L_{C}), which implies that ν^{∗}(M_{L}) = M_{L_{C}}. Tensoring the usual exact sequence on X

0 ⟶ O_{X} ⟶ ν_{∗}O_{C} ⟶ ν_{∗}O_{C}/O_{X} ⟶ 0,

by the line bundle ∧^{a}M_{L} ⊗ L^{(i−a)} ⊗ ω_{X} ⊗ η, we find that a sufficient condition for the vanishing H^{1}(X, ∧^{a}M_{L} ⊗ L^{⊗(i−a)} ⊗ ω_{X} ⊗ η) = 0 to hold, is to show that

H^{1}(C, ∧^{a}M_{L_{C}} ⊗ L_{C}^{⊗(i−a)} ⊗ K_{C} ⊗ η_{C}) = H^{1}(C, ∧^{a}M_{L_{C}} ⊗ L_{C}^{⊗(i−a)} ⊗ K_{C}(y + q) ⊗ η_{C}) = 0.

Since i < r, this follows directly from Proposition 2.4.

We are left with the case when [X, η, β] ∈ Δ_{0}^{ram}, when X := C ∪_{{q,y}} E, with E being a smooth rational curve, L_{C} ∈ W_{d}^{r}(C), L_{E} = O_{E} and η_{C}^{⊗2} = O_{C}(−y − q). We also have that M_{L|C} = M_{L_{C}} and M_{L|E} = H^{0}(C, L_{C} ⊗ O_{C}(−y − q)) ⊗ O_{E}. A standard argument involving the Mayer-Vietoris sequence on X shows that the vanishing of the group H^{1}(X, ∧^{a}M_{L}⊗L^{⊗(i−a)}⊗ω_{X} ⊗η) is implied by the following vanishing conditions

H^{1}(C, ∧^{a}M_{L_{C}} ⊗ L_{C}^{⊗(i−a)} ⊗ K_{C}(y + q) ⊗ η_{C}) = H^{1}(C, ∧^{a}M_{L_{C}} ⊗ L_{C}^{⊗(i−a)} ⊗ K_{C} ⊗ η_{C}) = 0.

The conditions of Proposition 2.4 being satisfied (i ≤ r − 1), we finish the proof. □

Proposition 2.5 enables us to define a sequence of tautological vector bundles on 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}): First, we set H := f′_{∗}(L). By Grauert’s theorem, it follows that H is a vector bundle of rank r + 1 with fibre H[X, η, β, L] = H^{0}(X, L). For j ≥ 0 we set

A_{0,j} := f′_{∗}(L^{⊗j} ⊗ ω_{f′} ⊗ P_{d}^{r}).

Since R^{1}f′_{∗}(L^{⊗j} ⊗ ω_{f′} ⊗ P_{d}^{r}) = 0 we find that A_{0,j} is a vector bundle over 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}) of rank equal to h^{0}(X, L^{⊗j} ⊗ω_{X} ⊗η) = jd+g −1. Next we introduce the global Lazarsfeld vector bundle M over X_{d}^{r} by the exact sequence

0 ⟶ M ⟶ f^{′∗}(H) ⟶ L ⟶ 0,

hence M_{f^{′−1}[X,η,β,L]} = M_{L} for each [X, η, β, L] ∈ 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}). Then for integers a, j ≥ 1 we define the sheaf

A_{a,j} := f′_{∗}(∧^{a}M ⊗ L^{⊗j} ⊗ ω_{f′} ⊗ P_{d}^{r}).

For each 1 ≤ a ≤ i − 1, we have proved that R^{1}f′_{∗}(∧^{a}M ⊗ L^{⊗(i−a)} ⊗ ω_{f′} ⊗ P_{d}^{r}) = 0 (cf. Proposition 2.5), therefore A_{a,i−a} is a vector bundle over 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}) having rank

rk(A_{a,i−a}) = χ(X, ∧^{a}M_{L} ⊗ L^{⊗(i−a)} ⊗ ω_{X} ⊗ η) = \binom{r}{a}k(i − a)(r + 1).

Proposition 2.5 also shows that for all integers 1 ≤ a ≤ i − 1, the vector bundles A_{a,i−a} sit in exact sequences

(6)  0 ⟶ A_{a,i−a} ⟶ ∧^{a}H ⊗ A_{0,i−a} ⟶ A_{a−1,i−a+1} ⟶ 0.

We shall need the expression for the Chern numbers of A_{a,i−a}. Using (6) it will be sufficient to compute c_{1}(A_{0,j}) for all j ≥ 0.

Proposition 2.6. For all j ≥ 0 one has the following formula in A^{1}(𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0})):

c_{1}(A_{0,j}) = λ + \frac{j}{2}B + \frac{j^{2}}{2} A − \frac{1}{4}δ_{0}^{ram}.

Proof. We apply Grothendieck-Riemann-Roch to the morphism f′ : X_{d}^{r} → 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}):

c_{1}(A_{0,j}) = c_{1}(f′_{!}(ω_{f′} ⊗ L^{⊗j} ⊗ P_{d}^{r})) =

= f′_{∗}[(1+c_{1}(ω_{f′}⊗L^{⊗j}⊗P_{d}^{r})+ \frac{c_{1}^{2}(ω_{f′} ⊗ L^{⊗j} ⊗ P_{d}^{r})}{2})(1− \frac{c_{1}(ω_{f′})}{2} + \frac{c_{1}^{2}(ω_{f′}) + [Sing(f′)]}{12})]_{2},

where Sing(f′) ⊂ X_{d}^{r} denotes the codimension 2 singular locus of the morphism f′, therefore f′_{∗}[Sing(f′)] = Δ′_{0} +Δ0^{′′} +2Δ_{0}^{ram}. We finish the proof using Mumford’s formula κ_{1} = f′_{∗}(c_{1}^{2}(ω_{f′})) = 12λ − (δ′_{0} + δ0^{′′} + 2δ_{0}^{ram}) and noting that f′_{∗}(c_{1}(L) · c_{1}(P_{d}^{r})) = 0 (the restriction of L to the exceptional divisor of f′ : X_{d}^{r} → 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}) is trivial) and f′_{∗}(c_{1}(ω_{f′}) · c_{1}(P_{d}^{r})) = 0,. Finally, according to Proposition 1.6 we have that f′_{∗}(c_{1}^{2}(P_{d}^{r})) = −δ_{0}^{ram}/2. □

Remark 2.7. While the construction of the vector bundles A_{a,j} depends on the choice of the Poincaré bundle L and that of the Prym bundle P_{d}^{r}, it is easy to check that if we set the vector bundles A := ∧^{i}H ⊗ A_{0,0} and B := A_{i−1,i}, then the vector bundle Hom(A, B) on 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}), as well as the morphism

ϕ ∈ H^{0}(𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}), Hom(A, B))

whose degeneracy locus is the virtual divisor \overline{D}_{g:k}, are independent of such choices. More precisely, let us denote by Ξ the collection of triples α := (π_{α}, L_{α}, (P_{d}^{r})_{α}), where π_{α} : Σ_{α} → 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}) is an étale surjective morphism from a scheme Σ_{α}, (P_{d}^{r})_{α} is a Prym bundle and L_{α} is a Poincaré bundle on p_{2,α} : X_{d}^{r} ×_{𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0})} Σ_{α} → Σ_{α}. Recall that

if Σ → 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}) is an étale surjection from a scheme and L and L′ are two Poincaré bundles on p_{2} : X_{d}^{r} ×_{𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0})} Σ → Σ, then the sheaf N := p_{2∗}Hom(L, L′) is invertible

and there is a canonical isomorphism L ⊗ p_{2}^{∗}N ≅ L′. For every α ∈ Ξ we construct the morphism between vector bundles of the same rank ϕ_{α} : A_{α} → B_{α} as above. Then since a straightforward cocycle condition is met, we find that there exists a vector bundle Hom(A, B) on 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}) together with a section ϕ ∈ H^{0}(𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}), Hom(A, B)) such that for every α = (π_{α}, L_{α}, (P_{d}^{r})_{α}) ∈ Ξ we have that

π_{α}^{∗}(Hom(A, B)) = Hom(A_{α}, B_{α}) and π_{α}^{∗}(ϕ) = ϕ_{α}.

We are finally in a position to compute the class of the divisor \overline{D}_{g:k}. Theorem 2.8. We fix integers k ≥ 2, b ≥ 0 and set i := kb − b + k − 2, r := kb + k − 2, g := ik + 1, d := rk as above. Then there exists a morphism ϕ : ∧^{i}H ⊗ A_{0,0} → A_{i−1,1} between vector bundles of the same rank over 𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}), such that the push-forward under σ of the restriction to 𝔊_{d}^{r}(R_{g}^{0}/M_{g}^{0}) of the degeneration locus of ϕ is precisely the effective divisor D_{g:k}. Moreover we have the following expression for its class in A^{1}(\overline{R}_{g}^{0}):

σ_{∗}(c_{1}(A_{i−1,1}−∧^{i}H⊗A_{0,0})) ≡ \binom{r}{b}\frac{N}{(r + k)}(kr+k−r−3)(𝔄λ− \frac{𝔅_{0}}{6} (δ′_{0}+δ0^{′′})− \frac{𝔅_{0}^{ram}}{12} δ_{0}^{ram}),

where

𝔄 = (k^{5} − 4k^{4} + 5k^{3} − 2k^{2})b^{3} + (3k^{5} − 13k^{4} + 24k^{3} − 23k^{2} + 9k)b^{2}+
+(3k^{5} − 14k^{4} + 34k^{3} − 45k^{2} + 24k − 4)b + k^{5} − 5k^{4} + 15k^{3} − 25k^{2} + 16k − 2,
𝔅_{0} = (k^{5} − 4k^{4} + 5k^{3} − 2k^{2})b^{3} + (3k^{5} − 13k^{4} + 22k^{3} − 17k^{2} + 5k)b^{2}+
+(3k^{5} − 14k^{4} + 30k^{3} − 33k^{2} + 14k − 2)b + k^{5} − 5k^{4} + 13k^{3} − 19k^{2} + 10k

and

𝔅_{0}^{ram} = (4k^{5} − 16k^{4} + 20k^{3} − 8k^{2})b^{3} + (12k^{5} − 52k^{4} + 85k^{3} − 65k^{2} + 20k)b^{2}+ +(12k^{5} − 56k^{4} + 111k^{3} − 114k^{2} + 53k − 8)b + 4k^{5} − 20k^{4} + 46k^{3} − 58k^{2} + 34k − 6.

Proof. To compute the class of the degeneracy locus of ϕ we use the exact sequence (6) and Proposition 2.6. We write the following identities in A^{1}(𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0})):

c_{1}(A_{i−1,1} − ∧^{i}H ⊗ A_{0,0}) = ∑_{l=0}^{i}(−1)^{l−1}c_{1}(∧^{i−l}H ⊗ A_{0,l}) =

= ∑_{l=0}^{i}(−1)^{l−1}((ld + g − 1)\binom{r}{i − l − 1}c_{1}(H) + \binom{r + 1}{i − l}c_{1}(A_{0,l})) =

= −k\binom{kb + k − 4}{b − 1}c_{1}(H) + \frac{1}{2}\binom{kb + k − 3}{b} 𝔟−

−\binom{kb + k − 2}{b}λ − \frac{kb + k − 2b − 3}{2(kb + k − 3)}\binom{kb + k − 3}{b} 𝔞 + \frac{1}{4}\binom{kb + k − 2}{b}δ_{0}^{ram} =

= \binom{r − 1}{b}(−\frac{kb}{r − 1} c_{1}(H) + \frac{1}{2} 𝔟 − \frac{r − 2b − 1}{2(r − 1)} 𝔞 − \frac{r}{r − b} λ + \frac{r}{4(r − b)}δ_{0}^{ram}),

where δ_{0}^{ram} = σ^{∗}(δ_{0}^{ram}) ∈ A^{1}(𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0})). The classes 𝔞, 𝔟 ∈ A^{1}(𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0})) and the line bundle H ∈ Pic(𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0})) are defined in terms of a Poincaré bundle L: If L′ := L⊗f^{′∗}(M) is another Poincaré bundle with M ∈ Pic(𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0})) and if 𝔞′, 𝔟′, H′ denote the classes defined in terms of L′ using (5), then we have formulas:

𝔞′ = 𝔞 + 2dc_{1}(M), 𝔟′ = 𝔟 + (2g − 2)c_{1}(M) and c_{1}(H′) = c_{1}(H) + (r + 1)c_{1}(M).

A straightforward calculation shows that the class

(7)  Ξ := −\frac{kb}{r − 1} c_{1}(H) + \frac{1}{2} 𝔟 − \frac{r − 2b − 1}{2(r − 1)} 𝔞 ∈ A^{1}(𝔊_{d}^{r}(\overline{R}_{g}^{0}/\overline{M}_{g}^{0}))

is independent of the choice of L and σ_{∗}(Ξ) = π^{∗}((σ_{0})_{∗}(Ξ_{0})), where the Ξ_{0} ∈ A^{1}(𝔊_{d}^{r}) is defined by the same formula (7) but inside Pic(𝔊_{d}^{r}). We outline below the computation of π^{∗}((σ_{0})_{∗}(Ξ_{0})) which uses [F2] in an essential way.

We follow closely [F2] and denote by \overline{M}_{g}^{1} := M_{g}^{0}∪Δ_{0}^{0}∪Δ_{1}^{0} the partial compactification of M_{g}^{0} obtained from \overline{M}_{g}^{0} by adding the stack Δ_{1}^{0} ⊂ Δ_{1} consisting of curves [C ∪_{y} E], where [C, y] ∈ M_{g−1,1} is a Brill-Noether general pointed curve and [E, y] ∈ \overline{M}_{1,1}. We extend σ_{0} : 𝔊_{d}^{r} → \overline{M}_{g}^{0} to a proper map σ_{1} : 𝔊̃_{d}^{r} → \overline{M}_{g}^{1} from the Deligne-Mumford stack of limit linear series 𝔤_{d}^{r} (cf. [EH], [F2], [Kh]). Then for each n ≥ 1 we consider the vector bundles G_{0,n} over 𝔊̃_{d}^{r} defined in [F2] Proposition 2.8 and which has the following description of its fibres:

• G_{0,n}(C, L) = H^{0}(C, L^{⊗n}), for each [C] ∈ M_{g}^{0} and L ∈ W_{d}^{r}(C).

• G_{0,n}(t) = H^{0}(C, L^{⊗n}(−y − q)) ⊕ ℂ · u^{n} ⊂ H^{0}(C, L^{⊗n}), where the point t = (C_{yq}, L ∈ W_{d}^{r}(C)) ∈ σ_{0}^{−1}([C_{yq}]), with u ∈ H^{0}(C, L) being a section such that

H^{0}(C, L) = H^{0}(C, L(−y − q)) ⊕ ℂ · u.

• G_{0,n}(t) = H^{0}(C, L^{⊗n}(−2y)) ⊕ ℂ · u^{n} ⊂ H^{0}(C, L^{⊗n}), where t = (C ∪_{y} E, l_{C}, l_{E}) ∈ σ_{0}^{−1}([C ∪_{y} E]) and (l_{C}, l_{E}) ∈ G_{d}^{r}(C) × G_{d}^{r}(E) being a limit linear series 𝔤_{d}^{r} with

l_{C} = (L, H^{0}(C, L)) and u ∈ H^{0}(C, L) a section such that
H^{0}(C, L) = H^{0}(C, L(−2y)) ⊕ ℂ · u.

We extend the classes 𝔞, 𝔟 ∈ A^{1}(𝔊_{d}^{r}) over the stack 𝔊̃_{d}^{r} by choosing a Poincaré bundle over \overline{M}_{g,1}^{1} ×\overline{M}{1 \atop g} 𝔊̃_{d}^{r} which restricts to line bundles of bidegree (d, 0) on curves [C ∪_{y} E] ∈

Δ_{1}^{0}. Grothendieck-Riemann-Roch applied to the universal curve over 𝔊̃_{d}^{r} gives that

(8)  c_{1}(G_{0,n}) = λ − \frac{n}{2}𝔟 + \frac{n^{2}}{2} 𝔞 ∈ A^{1}(𝔊̃_{d}^{r}), for all n ≥ 2

while obviously σ^{∗}(G_{0,1}) = H. We now fix a general pointed curve [C, q] ∈ M_{g−1} and an elliptic curve [E, y] ∈ M_{1,1} and consider the test curves (see also [F2] p. 7)

C^{0} := \{C/y ∼ q\}_{y∈C} ⊂ Δ_{0}^{0} ⊂ \overline{M}_{g}^{1} and C^{1} := \{C ∪_{y} E\}_{y∈C} ⊂ Δ_{1}^{0} ⊂ \overline{M}_{g}^{1}.

For n ≥ 1, the intersection numbers C^{0} · (σ_{0})_{∗}(c_{1}(G_{0,n})) and C^{1} · (σ_{0})_{∗}(c_{1}(G_{0,n})) can be computed using [F2] Lemmas 2.6 and 2.13 and Proposition 2.12. Together with the relation (cf. [F2] p. 15 for details)

(σ_{0})_{∗}(c_{1}(G_{0,n}))_{λ} − 12(σ_{0})_{∗}(c_{1}(G_{0,n}))_{δ_{0}} + (σ_{0})_{∗}(c_{1}(G_{0,n}))_{δ_{1}} = 0,

this completely determine the classes (σ_{0})_{∗}(c_{1}(G_{0,n})) ∈ A^{1}(𝔊̃_{d}^{r}). Then using (8) we find

(σ_{0})_{∗}(𝔞) ≡ N(−\frac{rk(r^{2}k^{2} − 3r^{2}k + 3rk^{2} + 2r^{2} + 2k^{2} + 4k − 7rk − 4r − 10)}{(rk − r + k − 3)(rk − r + k − 2)}λ+

+\frac{rk(r^{2}k^{2} − 3r^{2}k + 3rk^{2} − 8rk + 2r^{2} + 2k^{2} + r − k − 3)}{6(rk − r + k − 3)(rk − r + k − 2)}δ_{0} + ⋯),

(σ_{0})_{∗}(𝔟) ≡ N(\frac{6rk}{rk − r + k − 2}λ − \frac{rk}{2(rk − r + k − 2)}δ_{0} + ⋯),

and this completes the computation of the class (σ_{0})_{∗}(Ξ) and finishes the proof. □

The rather unwieldy expressions from Theorem 2.8 simplify nicely when k = 2, 3 when we obtain Theorems 0.2 and 0.3. Proof of Theorem 0.1 when g = 2i+1. We construct an effective divisor on \overline{R}_{g} satisfying the inequalities (2) as follows: The pull-back to \overline{R}_{g} of the Harris-Mumford divisor \overline{M}_{g,i+1}^{1} of curves of genus 2i + 1 with a 𝔤_{i+1}^{1} is given by the formula: π^{∗}(\overline{M}_{g,i+1}^{1}) ≡

≡ \frac{(2i − 2)!}{(i + 1)!(i − 1)!}(6(i + 2)λ − (i + 1)(δ′_{0} + δ0^{′′} + 2δ_{0}^{ram}) − ∑_{j=1}^{i} 3j(g − j)(δ_{j} + δ_{g−j} + δ_{j:g−j})).

We split \overline{D}_{2i+1:2} into boundary components of compact type and their complement

\overline{D}_{2i+1:2} ≡ E + ∑_{j=1}^{i}(a_{j}δ_{j} + a_{g−j}δ_{g−j} + a_{j:g−j}δ_{j:g−j}),

where a_{j}, a_{g−j}, a_{j:g−j} ≥ 0 and Δ_{j}, Δ_{g−j}, Δ_{j:g−j} ⊊ supp(E) for 1 ≤ j ≤ i, we consider the following positive linear combination on \overline{R}_{g}:

A := \frac{i! (i − 1)!}{(2i − 1) (2i − 3)!} ·π^{∗}(\overline{M}_{2i+1,i+1}^{1})+4\frac{(i!)^{2}}{(2i)!} ·E ≡ \frac{4(3i + 5)}{i + 1} λ−2(δ′_{0}+δ0^{′′})−3δ_{0}^{ram}−⋯,

where each of the coefficients of δ_{j}, δ_{g−j} and δ_{j:g−j} in the expansion of A are at least

\frac{6(i − 1)j(2i + 1 − j)}{(2i − 1)(i + 1)} ≥ 2.

Since \frac{4(3i+5)}{i+1} < 13 for i ≥ 8, the conclusion now follows using (2). For i = 7 we find that A ≡ 13λ − 2(δ′_{0} + δ0^{′′}) − 3δ_{0}^{ram} − ⋯, hence κ(\overline{R}_{15}) ≥ 0. To obtain that κ(\overline{R}_{15}) ≥ 1, we use the fact that on \overline{M}_{15} there exists a Brill-Noether divisor other than \overline{M}_{15,8}^{1}, namely the divisor \overline{M}_{15,14}^{3} of curves [C] ∈ M_{15} with a 𝔤_{14}^{3}. This divisor has the same slope s(\overline{M}_{15,14}^{3}) = s(\overline{M}_{15,8}^{1}) = 27/4, but supp(\overline{M}_{15,14}^{3}) ≠ supp(\overline{M}_{15,8}^{1}). It follows that there exist constants α, β, γ, m ∈ ℚ_{>0} such that

α · E + β · π^{∗}(\overline{M}_{15,8}^{1}) ≡ α · E + γ · π^{∗}(\overline{M}_{15,14}^{3}) ∈ |mK_{\overline{R}_{15}}|.

Thus we have found distinct multicanonical divisors on \overline{M}_{15}, that is, κ(\overline{M}_{15}) ≥ 1. □

Remark 2.9. The same numerical argument shows that if one replaces \overline{M}_{15,8}^{1} with any divisor D ∈ Eff(\overline{M}_{15}) with s(D) < s(\overline{M}_{15,8}^{1}) = 27/4, then \overline{R}_{15} is of general type. Any counterexample to the Slope Conjecture on \overline{M}_{15} makes \overline{R}_{15} of general type.

3. Koszul cohomology of Prym canonical curves

We recall that for a curve C, a line bundle L ∈ Pic^{d}(C) and integers i, j ≥ 0, the Koszul cohomology group K_{i,j}(C, L) is obtained from the complex

∧^{i+1}H^{0}(L) ⊗ H^{0}(L^{⊗(j−1)}) ⟶^{d_{i+1,j−1}} ∧^{i}H^{0}(L) ⊗ H^{0}(L^{⊗j}) ⟶^{d_{i,j}} ∧^{i−1}H^{0}(L) ⊗ H^{0}(L^{⊗(j+1)}),

where the maps are the Koszul differentials (cf. [GL]). There is a well-known connection between Koszul cohomology groups and Lazarsfeld bundles. Assuming that L is globally generated, a diagram chasing argument involving exact sequences of the type

0 ⟶ ∧^{a}M_{L} ⊗ L^{⊗b} → ∧^{a}H^{0}(L) ⊗ L^{⊗b} ⟶ ∧^{a−1}M_{L} ⊗ L^{⊗(b+1)} ⟶ 0

for various a, b ≥ 0 , yields the following identification (see also [GL] Lemma 1.10)

(9)  K_{i,j}(C, L) = \frac{H^{0}(C, ∧^{i}M_{L} ⊗ L^{⊗j})}{Image\{∧^{i+1}H^{0}(C, L) ⊗ H^{0}(C, L^{⊗(j−1)})\}} .

We fix [C, η] ∈ R_{g}, set L := K_{C} ⊗ η ∈ W_{2g−2}^{g−2}(C) and consider the Prym-canonical map

C →^{|L|} P^{g−2}. We denote by I_{C} ⊂ O_{P}_{g−2} the ideal sheaf of the Prym-canonical curve.

By analogy with [F2] we study the Koszul stratification of R_{g} and define the strata

U_{g,i} := {[C, η] ∈ R_{g} : K_{i,2}(C, K_{C} ⊗ η) ≠ ∅}.

Using (9) we write the series of equivalences

[C, η] ∈ U_{g,i} ⇔ H^{1}(C, ∧^{i+1}M_{L} ⊗ L) ≠ ∅ ⇔ h^{0}(C, ∧^{i+1}M_{L} ⊗ L) >

> \binom{g − 2}{i + 1}(−\frac{(i + 1)(2g − 2)}{g − 2} + (g − 1)).

Next we write down the exact sequence

0 ⟶ H^{0}(∧^{i+1}M_{P}_{g−2}(1)) ⟶^{a} H^{0}(C, ∧^{i+1}M_{L} ⊗ L) ⟶ H^{1}(∧^{i+1}M_{P}_{g−2} ⊗ I_{C}(1)) ⟶ 0,

and then also

Coker(a) = H^{1}(P^{g−2}, ∧^{i+1}M_{P}_{g−2} ⊗ I_{C}(1)) = H^{0}(P^{g−2}, ∧^{i}M_{P}_{g−2} ⊗ I_{C}(2)).

Using the well-known fact that h^{0}(P^{g−2}, ∧^{i+1}M_{P}_{g−2}(1)) = \binom{g−1}{i+2} (use for instance the Bott vanishing theorem), we end-up with the following equivalence:

(10)  [C, η] ∈ U_{g,i} ⇔ h^{0}(P^{g−2}, ∧^{i}M_{P}_{g−2} ⊗ I_{C}(2)) > \binom{g − 3}{i}\frac{(g − 1)(g − 2i − 6)}{i + 2}.

Proposition 3.1. (1) For g < 2i + 6, we have that K_{i,2}(C, K_{C} ⊗ η) ≠ ∅ for any [C, η] ∈ R_{g},

that is, the Prym-canonical curve C ⟶^{|K_{C}+η|} P^{g−2} does not satisfy property (N_{i}).

(2) For g = 2i + 6, the locus U_{g,i} is a virtual divisor on R_{g}, that is, there exist vector bundles G_{i,2} and H_{i,2} over R_{g} such that rank(G_{i,2}) = rank(H_{i,2}), together with a bundle morphism ϕ : H_{i,2} → G_{i,2} such that U_{g,i} is the degeneracy locus of ϕ.

Proof. Part (1) is an immediate consequence of (10), since we have the equivalence

K_{i,2}(C, K_{C} ⊗ η) = 0 ⇔ h^{0}(P^{g−2}, ∧^{i}M_{P}_{g−2} ⊗ I_{C}(2)) = \binom{g − 3}{i}\frac{(g − 1)(g − 2i − 6)}{i + 2}.

For part (2) one constructs two vector bundles G_{i,2} and H_{i,2} over R_{g} having fibres

G_{i,2}[C, η] = H^{0}(C, ∧^{i}M_{K_{C}⊗η}(2)) and H_{i,2}[C, η] = H^{0}(P^{g−2}, ∧^{i}M_{P}_{g−2}(2)). There is a natural morphism ϕ : H_{i,2} → G_{i,2} given by restriction. We have that

rank(G_{i,2}) = \binom{g − 2}{i}(−\frac{i(2g − 2)}{g − 2} + 3(g − 1)) and rank(H_{i,2}) = (i + 1)\binom{g}{i + 2}

and the condition that rank(G_{i,2}) = rank(H_{i,2}) is equivalent to g = 2i + 6. □

We describe a set-up that will be used to define certain tautological sheaves over R̃_{g} and compute the class [\overline{U}_{g,i}]^{virt}. We use the notation from Subsection 1.1, in particular from Proposition 1.7 and recall that f : X → R̃_{g} is the universal Prym curve, P ∈ Pic(X) denotes the universal Prym line bundle and N_{i} = f_{∗}(ω_{f}^{⊗i} ⊗P^{⊗i}). We denote by T := E0^{′′} ∩ Sing(f) the codimension 2 subvariety corresponding to Wirtinger covers [C_{yq}, η ∈ Pic^{0}(C_{yq})[2], ν(y) = ν(q)] ∈ X (where ν^{∗}(η) = O_{C}), with the marked point being the node of the underlying curve C_{yq}. Let us fix a point [X := C_{yq}, η, β] ∈ Δ̃′_{0} ∪Δ̃0^{′′} where as usual ν : C → X is the normalization map. Then we have an identification

(11) N_{1}[X, η, β] = Ker{H^{0}(C, ω_{C}(y + q) ⊗ η_{C}) → (ν_{∗}O_{C}/O_{X}) ⊗ ω_{X} ⊗ η ≅ ℂ_{y∼q}},

where the map is given by taking the difference of residues at y and q. Note that when η_{C} = O_{C}, that is when[X, η, β] ∈ Δ̃0^{′′}, we have that N_{1}[X, η, β] = H^{0}(C, ω_{C}). For a point

[X = C ∪_{\{y,q\}} E, η_{C} ∈ \sqrt{O_{C}(−y − q)}, η_{E}] ∈ Δ̃_{0}^{ram}

we have an identification

(12) N_{1}[X, η, β] = Ker{H^{0}(C, ω_{C}(y + q) ⊗ η_{C}) ⊕ H^{0}(E, O_{E}(1)) → (ω_{X} ⊗ η)_{y,q} ≅ ℂ_{y,q}^{2}}.

We set

M := Ker{f^{∗}(N_{1}) → ω_{f} ⊗ P}.

From the discussion above it is clear that the image of f^{∗}(N_{1}) → ω_{f} ⊗ P is ω_{f} ⊗ P ⊗ I_{T}. Since T ⊂ X is smooth of codimension 2 it follows that M is locally free. For a, b ≥ 0, we define the sheaf E_{a,b} := f_{∗}(∧^{a}M ⊗ ω_{f}^{⊗b} ⊗ P^{⊗b}) over R̃_{g}. Clearly E_{a,b} is locally free. We have that E_{0,b} = N_{b} for b ≥ 0, and we always have left-exact sequences

(13)  0 ⟶ E_{a,b} ⟶ ∧^{a}E_{0,1} ⊗ E_{0,b} ⟶ E_{a−1,b+1},

which are right-exact off the divisor Δ̃0^{′′} (to be proved later). We then define inductively a sequence of vector bundles {H_{a,b}}_{a,b≥0} over R̃_{g} in the following way: We set H_{0,b} := Sym^{b}N_{1} for each b ≥ 0. Then having defined H_{a−1,b} for all b ≥ 0, we define the vector bundle H_{a,b} by the exact sequence

(14)  0 ⟶ H_{a,b} ⟶ ∧^{a}H_{0,1} ⊗ Sym^{b}H_{0,1} ⟶ H_{a−1,b+1} ⟶ 0.

For a point [X, η, β] ∈ R̃_{g}, if we use the identification H^{0}(X, ω_{X}⊗η) = H^{0}(P^{g−2}, O_{P}_{g−2}(1)), we have a natural identification of the fibre

H_{a,b}[X, η, β] = H^{0}(P^{g−2}, ∧^{a}M_{P}_{g−2}(b)).

By induction on a ≥ 0, there exist vector bundle morphisms ϕ_{a,b} : H_{a,b} → E_{a,b}. Proposition 3.2. For b ≥ 2 and a ≥ 0 we have the vanishing of the higher direct images

R^{1}f_{∗}(∧^{a}M ⊗ ω_{f}^{⊗b} ⊗ P^{⊗b})_{| R_{g}∪Δ̃′_{0}∪Δ̃_{0}^{ram}} = 0.

It follows that the sequences (13) are right-exact off the divisor Δ̃0^{′′} of R̃_{g}.

Proof. Over the locus R_{g} the vanishing is a consequence of Proposition 2.4. For simplicity we prove that R^{1}f_{∗}(∧^{a}M ⊗ ω_{f}^{⊗b} ⊗ P^{⊗b}) ⊗ OΔ̃_{0}^{ram} = 0, the vanishing over Δ̃′_{0} being similar. We fix a point [X = C ∪_{{y,q}} E, η_{C}, η_{E}] ∈ Δ̃_{0}^{ram}, with η_{C}^{⊗2} = O_{C}(−y − q), η_{E} = O_{E}(1) and set L := ω_{X} ⊗ η ∈ Pic^{2g−2}(X). We show that H^{1}(X, ∧^{a}M_{L} ⊗ L^{⊗b}) = 0 for all a ≥ 0 and b ≥ 2. A Mayer-Vietoris argument shows that it suffices to prove that

(15)  H^{1}(C, ∧^{a}M_{L} ⊗ L^{⊗b} ⊗ O_{C}) = 0, H^{1}(E, ∧^{a}M_{L} ⊗ L^{⊗b} ⊗ O_{E}) = 0, and
(16)  H^{1}(C, ∧^{a}M_{L} ⊗ L^{⊗b} ⊗ O_{C}(−y − q)) = 0.

For L_{C} := L ⊗ O_{C} = K_{C}(y + q) ⊗ η_{C} and L_{E} := L_{E} ⊗ O_{E}, we write the exact sequences

0 ⟶ H^{0}(C, L_{C}(−y − q)) ⊗ O_{E} ⟶ M_{L} ⊗ O_{E} ⟶ M_{L_{E}} ⟶ 0, and

0 ⟶ H^{0}(E, L_{E}(−y − q)) ⊗ O_{C} ⟶ M_{L} ⊗ O_{C} ⟶ M_{L_{C}} ⟶ 0,

and we find that M_{L} ⊗ O_{C} = M_{L_{C}} while obviously M_{L_{E}} = O_{E}(−1). We conclude that the statements (15) and (16) for all a ≥ 0 and b ≥ 2 can be reduced to showing that H^{1}(C, ∧^{a}M_{L_{C}} ⊗ L_{C}^{⊗b}) = H^{1}(C, ∧^{a}M_{L_{C}} ⊗ L_{C}^{⊗b} ⊗ O_{C}(−y − q)) = 0, for all a ≥ 0, b ≥ 2. This is now a direct application of Proposition 2.4. □

Proof of Theorem 0.6. We have constructed the vector bundle morphism ϕ_{i,2} : H_{i,2} → E_{i,2} over R̃_{g}. For g = 2i+6 we have that rank(H_{i,2}) = rank(E_{i,2}) and the virtual Koszul class [\overline{U}_{g,i}]^{virt} is given by c_{1}(E_{i,2} − H_{i,2}). We recall that for a rank e vector bundle E over a variety X and for i ≥ 1, we have the formulas c_{1}(∧^{i}E) = \binom{e−1}{i−1}c_{1}(E) and c_{1}(Sym^{i}(E)) = \binom{e+i−1}{e}c_{1}(E). Using (13) we find that there exists a constant α ≥ 0 such that

c_{1}(E_{i,2}) − α · δ0^{′′} = ∑_{l=0}^{i}(−1)^{l}c_{1}(∧^{i−l}E_{0,1} ⊗ E_{0,l+2}) = ∑_{l=0}^{i}(−1)^{l}\binom{g − 1}{i − l}c_{1}(E_{0,l+2})+

+ ∑_{l=0}^{i}(−1)^{l}((g − 1)(2l + 3))\binom{g − 2}{i − l − 1}c_{1}(E_{0,1}),

while a repeated application of the exact sequence (14) gives that

c_{1}(H_{i,2}) = ∑_{l=0}^{i}(−1)^{l}c_{1}(∧^{i−l}H_{0,1} ⊗ Sym^{l+2}H_{0,1}) =

= ∑_{l=0}^{i}(−1)^{l}(\binom{g − 1}{i − l}c_{1}(Sym^{l+2}(H_{0,1})) + \binom{g + l}{l + 2}c_{1}(∧^{i−l}H_{0,1}))

= ∑_{l=0}^{i}(−1)^{l}(\binom{g − 1}{i − l}\binom{g + l}{g − 1} + \binom{g + l}{l + 2}\binom{g − 2}{i − l − 1})c_{1}(H_{0,1}),

with E_{0,1} = H_{0,1} = N_{1} and E_{0,l+2} = N_{l+2} for l ≥ 0. Proposition 1.7 finishes the proof. □

Comparing these formulas to the canonical class of \overline{R}_{g}, one obtains that \overline{R}_{g} is of general type for g > 12.

4. Effective divisors on \overline{R}_{g}

We now use in an essential way results from [F3] to produce myriads of effective divisors on \overline{R}_{g}. This construction, though less explicit than that of \overline{U}_{2i+6} and \overline{D}_{g:k}, is still very effective and we use it to show \overline{R}_{18}, \overline{R}_{20} and \overline{R}_{22} are of general type.

We consider the morphism χ : \overline{R}_{g} → \overline{M}_{2g−1} given by χ([C, η]) := [C̃], where f : C̃ → C is the étale double cover determined by η. Thus one has

f_{∗}OC̃ = O_{C} ⊕η and H^{i}(C̃, f^{∗}L) = H^{i}(C, L)⊕H^{i}(C, L⊗η) for any L ∈ Pic(C), i = 0, 1.

The pullback map χ^{∗} at the level of Picard groups has been determined by M. Bernstein in [Be] Lemma 3.1.3. We record her results:

Proposition 4.1. The pullback map χ^{∗} : Pic(\overline{R}_{g}) → Pic(\overline{M}_{2g−1}) is given as follows:

χ^{∗}(λ) = 2λ−\frac{1}{4}δ_{0}^{ram}, χ^{∗}(δ_{0}) = δ_{0}^{ram}+2(δ′_{0}+δ0^{′′}+∑_{i=1}^{[g/2]} δ_{i:g−i}), χ^{∗}(δ_{i}) = 2δ_{g−i} for1 ≤ i ≤ g−1.

Proof. The formula for χ^{∗}(δ_{i}) when 1 ≤ i ≤ g − 1 is immediate. To determine χ^{∗}(λ) one notices that χ^{∗}((κ_{1})_{\overline{M}_{2g−1}}) = 2(κ_{1})_{\overline{R}_{g}} and the rest follows from Mumford’s formulas (κ_{1})_{\overline{M}_{2g−1}} = 12λ − δ ∈ Pic(\overline{M}_{2g−1}) and (κ_{1})_{\overline{R}_{g}} = 12λ − π^{∗}(δ) ∈ Pic(\overline{R}_{g}). □

We set the integer g′ := 1 + \frac{g−1}{g} \binom{2g}{g−1}. In [F3] we have studied the rational map

ϕ : \overline{M}_{2g−1} − − > \overline{M}_{1+ \frac{g−1}{g} \binom{2g}{g−1}}, ϕ[Y] := W_{g+1}^{1}(Y),

and determined the pullback map at the level of divisors ϕ^{∗} : Pic(\overline{M}_{g′}) → Pic(\overline{M}_{2g−1}). In particular, we proved that if A ∈ Pic(\overline{M}_{g′}) is a divisor of slope s(A) = s, then the slope of the pullback ϕ^{∗}(A) is equal to (cf. [F3] Theorem 0.2)

(17)  s(ϕ^{∗}(A)) = 6 + \frac{8g^{3}s − 32g^{3} − 19g^{2}s + 66g^{2} + 6gs − 16g + 3s + 6}{(g − 1)(g + 1)(g^{2}s − 2gs − 4g^{2} + 7g + 3)}.

To obtain effective divisors of small slope on \overline{R}_{g} we shall consider pullbacks (ϕχ)^{∗}(A), where A ∈ Ample(\overline{M}_{g′}). (Of course, one can consider the cone χ^{∗}(Ample(\overline{M}_{2g−1})), but a quick look at Proposition (4.1) shows that it is impossible to obtain in this way divisors on \overline{R}_{g} satisfying the inequalities (2). Pulling back merely effective divisors \overline{M}_{2g−1} rather than ample ones, is problematic since χ(\overline{R}_{g}) tends to be contained in many geometric divisors on \overline{M}_{2g−1}.) In order for the pullbacks χ^{∗}ϕ^{∗}(A) to be well-defined as effective divisors on \overline{R}_{g} we prove the following result:

Proposition 4.2. If dom(ϕ) ⊂ \overline{M}_{2g−1} is the domain of definition of the rational morphism ϕ : \overline{M}_{2g−1} → \overline{M}_{g′}, then χ(\overline{R}_{g}) ∩ dom(ϕ) ≠ ∅. It follows that for any ample divisor A ∈ Ample(\overline{M}_{g′}), the pullback χ^{∗}ϕ^{∗}(A) ∈ Eff(\overline{R}_{g}) is well-defined.

Proof. We take a general point [C ∪_{y} E, η_{C} = O_{C}, η_{E}] ∈ Δ_{1} ⊂ \overline{R}_{g}. The corresponding admissible double cover is then f : C_{1} ∪_{y_{1}} Ẽ ∪_{y_{2}} C_{2} → C ∪_{y} E, where [C_{1}, y_{1}] and [C_{2}, y_{2}] are copies of [C, y] mapping isomorphically to [C, y], and f : Ẽ → E is the étale double cover induced by the torsion point η_{E} ∈ Pic^{0}(E)[2]. We have that C_{i} ∩ Ẽ = {y_{i}}, where fẼ(y_{1}) = fẼ(y_{2}) = y. Thus χ[C ∪ E, O_{C}, η_{E}] := [C_{1} ∪_{y_{1}} Ẽ ∪_{y_{2}} C_{2}], where y_{1}, y_{2} ∈ Ẽ are such that OẼ(y_{1} − y_{2}) is a 2-torsion point in Pic^{0}(Ẽ).

Suppose now that X := C_{1} ∪_{y_{1}} E ∪_{y_{2}} C_{2} is a curve of compact type such that

[C_{i}, y_{i}] ∈ M_{g−1,1} (i = 1, 2) and [E, y_{1}, y_{2}] ∈ M_{1,2} are all Brill-Noether general. In particular, the class y_{1} − y_{2} ∈ Pic^{0}(E) is not torsion. Then ϕ([X]) := [\overline{W} _{g+1}^{1}(X)] is the variety of limit linear series 𝔤_{g+1}^{1} on X. The general point of each irreducible component of \overline{W} _{g+1}^{1}(X) corresponds to a refined linear series l on X satisfying the following compatibility conditions in terms of Brill-Noether numbers (see also [EH], [F3]): (18) 1 = ρ(l_{C_{1}}, y_{1}) + ρ(l_{C_{2}}, y_{2}) + ρ(l_{E}, y_{1}, y_{2}) = 1 and ρ(l_{C_{1}}, y_{1}), ρ(l_{C_{2}}, y_{2}), ρ(l_{E}, y_{1}, y_{2}) ≥ 0.

If ρ(l_{C_{2}}, y_{2}) = 1, we find two types of components of \overline{W} _{g+1}^{1}(X) which we describe: Since ρ(l_{C_{1}}, y_{1}) = 0, there exists an integer 0 ≤ a ≤ g/2 such that a^{l_{C_{1}}} (y_{1}) = (a, g + 2 − a). On E there are two choices for l_{E} ∈ G_{g+1}^{1}(E) such that a^{l_{E}}(y_{1}) = (a − 1, g + 1 − a). Either a^{l_{E}}(y_{2}) = (a, g +1−a) (there is a unique such l_{E}), and then l_{C_{2}} belongs to the connected curve T_{a} := {l_{C_{2}} ∈ G_{g+1}^{1}(C_{2}) : a^{l_{C_{2}}} (y_{2}) ≥ (a, g+1−a)}, or else, a^{l_{E}}(y_{2}) = (a−1, g+2−a) (again, there is a unique such l_{E}), and then the C_{2}-aspect of l belongs to the curve T′_{a} := {l_{C_{2}} ∈ G_{g+1}^{1}(C_{2}) : a^{l_{C_{2}}} (y_{2}) ≥ (a − 1, g + 2 − a)}. Thus {l_{C_{1}}} × T_{a} and {l_{C_{2}}} × T′_{a} are irreducible components of \overline{W} _{g+1}^{1}(X). When ρ(l_{E}, y_{1}, y_{2}) = 1, then there are three types of irreducible components of \overline{W} _{g+1}^{1}(X) corresponding to the cases

a^{l_{E}}(y_{1}) = (a − 1, g + 1 − a), a^{l_{E}}(y_{2}) = (a − 1, g + 1 − a) for 0 ≤ a ≤ g/2,
a^{l_{E}}(y_{1}) = (a − 1, g + 1 − a), a^{l_{E}}(y_{2}) = (a, g − a) for 1 ≤ a ≤ (g − 1)/2, and
a^{l_{E}}(y_{1}) = (a − 1, g + 1 − a), a^{l_{E}}(y_{2}) = (a − 2, g + 2 − a) for 2 ≤ a ≤ (g − 1)/2.

Finally, the case ρ(l_{C_{1}}, y_{1}) = 1 is identical to the case ρ(l_{C_{2}}, y_{2}) = 1 by reversing the role of the curves C_{1} and C_{2}. The singular points of \overline{W} _{g+1}^{1}(X) correspond to (necessarily) crude limit 𝔤_{g+1}^{1}’s satisfying ρ(l_{C_{1}}, y_{1}) = ρ(l_{C_{2}}, y_{2}) = ρ(l_{E}, y_{1}, y_{2}) = 0. For such l, there must exist two irreducible components of X, say Y and Z, for which Y ∩ Z = {x} and such that a_{0}^{l_{Y}} (x) + a_{1}^{l_{Z}}(x) = g + 2 and a_{1}^{l_{Y}} (x) + a_{0}^{l_{Z}}(x) = g + 1. The point l lies precisely on the two irreducible components of \overline{W} _{g+1}^{1}(X): The one corresponding to refined limit 𝔤_{g+1}^{1} with vanishing sequence on Y equal to (a_{0}^{l_{Y}} (x) − 1, a_{1}^{l_{Y}} (x)), and the one with vanishing (a_{0}^{l_{Z}}(x), a_{1}^{l_{Z}}(x)−1) on Z. Thus \overline{W} _{g+1}^{1}(X) is a stable curve of compact type, so [X] ∈ dom(ϕ). Using [F3], this set-theoretic description applies to the image under ϕ of any point [C_{1} ∪_{y_{1}} E ∪_{y_{2}} C_{2}], in particular to [C_{1} ∪_{y_{1}} Ẽ ∪_{y_{2}} C_{2}] = χ([C ∪_{y} E]). We have showed that χ(Δ_{1}) ∩ dom(ϕ) ≠ ∅. □

Proof of Theorem 0.1 for genus g = 18, 20, 22. We construct an effective divisor on \overline{R}_{g} which satisfies the inequalities (2) and which is of the form

μπ^{∗}(D) + ϵχ^{∗}ϕ^{∗}(A) = αλ − 2(δ′_{0} + δ0^{′′}) − 3δ_{0}^{ram} − ∑_{i=1}^{[g/2]}(b_{i}δ_{i} + b_{g−i}δ_{g−i} + b_{i:g−i}δ_{i:g−i}),

where A ≡ sλ − δ ∈ Pic(\overline{M}_{g′}) is an ample class (which happens precisely when s > 11, cf. [CH]), D ∈ Eff(\overline{M}_{g}) and μ, ϵ > 0 and α < 13. We solve this linear system using Proposition 4.1 and find that we must have

ϵ = \frac{8}{12 − s(ϕ^{∗}(A))} and μ = \frac{16 − 2s(ϕ^{∗}(A))}{12 − s(ϕ^{∗}(A))}.

To conclude that \overline{R}_{g} is of general type, it suffices to check that the inequality

α = \frac{8s(ϕ^{∗}(A))}{12 − s(ϕ^{∗}(A))} + (6 + \frac{12}{g + 1})\frac{16 − 2s(ϕ^{∗}(A))}{12 − s(ϕ^{∗}(A))} < 13

has a solution s = s(A) ≥ 11. Using (17), we find that this is the case for g ≥ 18. □

5. The enumerative geometry of \overline{R}_{g} in small genus

In this Section we describe the divisors D_{g:k} and U_{g,i} for small g. We start with the case g = 3. This result has been first obtained by M. Bernstein [Be] Theorem 3.2.3 using test curves inside \overline{R}_{3}. Our method is more direct and uses the identification of cycles C − C = Θ_{Q_{C}} ⊂ Pic^{0}(C), valid for all curves [C] ∈ M_{3}.

Theorem 5.1. The divisor D_{3:2} = {[C, η] ∈ R_{3} : η ∈ C − C} is equal to the locus of étale

double covers [C̃ →^{f} C] ∈ R_{3} such that [C̃] ∈ M_{5} is hyperelliptic. We have the equality of

cycles \overline{D}_{3:2} ≡ 8λ − δ′_{0} − 2δ0^{′′} − \frac{3}{2}δ_{0}^{ram} − 6δ_{1} − 4δ_{2} − 2δ_{1:2} ∈ Pic(\overline{R}_{3}). Moreover,

π_{∗}(\overline{D}_{3:2}) ≡ 56 · \overline{M}_{3,2}^{1} = 56 · (9λ − δ_{0} − 3δ_{1}) ∈ Pic(\overline{M}_{3}).

This equality corresponds to the fact that for an étale double cover f : C̃ → C, the source C̃ is hyperelliptic if and only if C is hyperelliptic and η ∈ C − C ⊂ Pic^{0}(C).

Proof. We use the set-up from Theorem 2.8 and recall that there exists a vector bundle morphism ϕ : H ⊗ A_{0,0} → A_{0,1} over \overline{R}_{3}^{0} such that Z_{1}(ϕ) ∩ R_{3} = D_{3:2}. Here H = π^{∗}(𝔼), A_{0,0}[X, η, β] = H^{0}(X, ω_{X} ⊗ β) and A_{0,1}[X, η, β] = H^{0}(X, ω_{X}^{⊗2} ⊗ β), for each point [X, η, β] ∈ R̃_{g}. Using (11) and (12) we check that both ϕ_{|Δ′_{0}} and ϕ_{|Δ_{0}^{ram}} are generically non-degenerate. Over a point t = [C_{yq}, η, β] ∈ Δ0^{′′} corresponding to a Wirtinger covering (i.e. ν : C → C_{yq}, with [C] ∈ M_{2} and ν^{∗}(η) = O_{C}), we have that

ϕ(t) : H^{0}(C, K_{C}) ⊗ H^{0}(C, K_{C} ⊗ O_{C}(y + q)) → A_{0,1}(t) ⊂ H^{0}(C, ω_{C}^{⊗2} ⊗ O_{C}(2y + 2q)).

From the base point free pencil trick we find that Ker(ϕ(t)) = H^{0}(C, O_{C}(y + q)), that is, ϕ_{|Δ′′_{0}} is everywhere degenerate and the class c_{1}(A_{0,1} − H ⊗ A_{0,0}) − δ0^{′′} ∈ Pic(\overline{R}_{3}^{0}) is effective. From the formulas π_{∗}(λ) = 63λ, π_{∗}(δ′_{0}) = 30δ_{0}, π_{∗}(δ0^{′′}) = δ_{0} and π_{∗}(δ_{0}^{ram}) = 16δ_{0}, we obtain that

s(π_{∗}(c_{1}(A_{0,1} − H ⊗ A_{0,0}) − δ0^{′′})) = 9.

The hyperelliptic locus \overline{M}_{3,2}^{1} is the only divisor on D ∈ Eff(\overline{M}_{3}) with Δ_{i} ⊊ supp(D) for i = 0, 1 and s(D) ≤ 9, which leads to the formula π_{∗}(\overline{D}_{3:2}) = 56 · \overline{M}_{3,2}^{1}. □

Theorem 5.2. The divisor \overline{D}_{5:2} := \{[C, η] ∈ R_{5} : η ∈ C_{2} − C_{2}\} equals the locus of étale

double covers [C̃ →^{f} C] ∈ R_{5} such that the genus 9 curve C̃ is tetragonal. We have the formula

\overline{D}_{5:2} = 14λ − 2(δ′_{0} + δ0^{′′}) − \frac{5}{2}δ_{0}^{ram} − 10δ_{4} − 4δ_{1:4} − ⋯ ∈ Pic(\overline{R}_{5}).

Proof. We start with an étale cover f : C̃ →^{2:1} C corresponding to the torsion point η = O_{C}(D − E), with D, E ∈ C_{2}. Then

H^{0}(C̃, OC̃(f^{∗}D)) = H^{0}(C, O_{C}(D)) ⊕ H^{0}(C, O_{C}(E)),

that is, |f^{∗}D| ∈ G_{4}^{1}(C̃) and [C̃] ∈ \overline{M}_{9,4}^{1}. Conversely, if l ∈ G_{4}^{1}(C̃), then l must be invariant under the involution of C̃ and then f_{∗}(l) ∈ G_{4}^{1}(C) contains two divisors of the type 2x + 2y ≡ 2p + 2q. Then we take η = O_{C}(x + y − p − q), that is, [C, η] ∈ D_{5:2}. □

Remark 5.3. Since codim(\overline{M}_{9,4}^{1}, \overline{M}_{9}) = 3 while D_{5:2} is a divisor in R_{3}, there seems to be a dimensional discrepancy in Theorem 5.2. This is explained by noting that for an étale double covering f : C̃ → C over a general curve [C] ∈ M_{5}, the codimension 1 condition gon(C̃) ≤ 5 is equivalent to the seemingly stronger condition gon(C̃) ≤ 4. Indeed, if l ∈ G_{5}^{1}(C̃) is base point free, then l is not invariant under the involution of C̃ and dim |f_{∗}l| ≥ 2 so G_{5}^{2}(C) ≠ ∅, a contradiction with the genericity assumption on C.

Theorem 5.4. The divisor D_{4:3} = {[C, η] ∈ R_{4} : ∃A ∈ W_{3}^{1}(C) withH^{0}(C, A ⊗ η) ≠ 0} can be identified with the locus of Prym curves [C, η] ∈ R_{4} such that the Prym-canonical model

C ⟶^{|K_{C}⊗η|} P^{2} is a plane sextic curve with a triple point. We also have the class formula

\overline{D}_{4:3} ≡ 8λ − δ′_{0} − 2δ0^{′′} − \frac{7}{4}δ_{0}^{ram} − 4δ_{3} − 7δ_{1} − 3δ_{1:3} − ⋯ ∈ Pic(\overline{R}_{4}),

hence π_{∗}(\overline{D}_{4:3}) = 60 · \overline{GP}_{4,3}^{1} = 60(34λ − 4δ_{0} − 14δ_{1} − 18δ_{2}) ∈ Pic(\overline{M}_{4}), where

GP_{4,3}^{1} ⊂ M_{4} := {C] ∈ M_{4} : ∃A ∈ W_{3}^{1}(C), A^{⊗2} = K_{C}}

is the Gieseker-Petri divisor of curves [C] ∈ M_{4} with a vanishing theta-null.

Proof. We start with a Prym curve [C, η] ∈ R_{4} such that there exists A ∈ W_{3}^{1}(C) with H^{0}(C, A ⊗ η) ≠ 0. We claim that A^{⊗2} = K_{C}, that is, [C] ∈ GP_{4,3}^{1}. Indeed, assuming the opposite, we find disjoint divisors D_{1}, D_{2} ∈ C_{3} such that D_{1} ∈ |A ⊗ η| and D_{2} ∈ |K_{C} ⊗ A^{∨} ⊗ η|. In particular, the subspaces H^{0}(C, K_{C} ⊗ η(−D_{i})) ⊂ H^{0}(C, K_{C}) are both of dimension 2, hence they intersect non-trivially, that is H^{0}(C, K_{C} ⊗η(−D_{1} −D_{2})) ≠ 0. Since D_{1} + D_{2} ≡ K_{C}, this implies η = 0, a contradiction.

The proof that the vector bundle morphism ϕ : H ⊗ A_{0,0} → A_{0,1} constructed in the proof of Theorem 2.8 is degenerate with order 1 along the divisor Δ0^{′′} ⊂ \overline{R}_{4} follows from (11). Thus c_{1}(A_{0,1} − H ⊗ A_{0,0}) − δ0^{′′} ∈ Pic(\overline{R}_{4}) is an effective class and its pushforward to \overline{M}_{4} has slope 17/2. The only divisor D ∈ Eff(\overline{M}_{4}) with Δ_{i} ⊊ supp(D) for i = 0, 1, 2 and s(D) ≤ 17/2, is the theta-null divisor \overline{GP}_{4,3}^{1} (cf. [F3] Theorem 5.1). □

Remark 5.5. For a general point [C, η] ∈ R_{4}, the Prym-canonical curve ι : C ⟶^{|K_{C}⊗η|} P^{2} is a plane sextic with 6 nodes which correspond to the preimages in ϕ^{−1}(η) under the second difference map

C_{2} × C_{2} → Pic^{0}(C), (D_{1}, D_{2}) ↦ O_{C}(D_{1} − D_{2}).

Note that W_{2}(C) · (W_{2}(C) + η) = 6. For a general [C, η] ∈ D_{4:3}, the model ι(C) ⊂ P^{2} has a triple point. For a hyperelliptic curve [C] ∈ M_{4,2}^{1}, out of the 255 = 2^{2g} − 1 étale

double covers of C, there exist 210 for which C ⟶^{|K_{C}⊗η|} P^{2} has an ordinary 4-fold point and no other singularity. The remaining 45 = \binom{2g+2}{2} coverings correspond to the case η = O_{C}(x − y), with x, y ∈ C being Weierstrass points, when |K_{C} ⊗ η| has 2 base points and ι is a degree 2 map onto a conic.

6. The singularities of the moduli space of Prym curves

The moduli space \overline{R}_{g} is a normal variety with finite quotient singularities. To determine its Kodaira dimension we consider a smooth model R̂_{g} of \overline{R}_{g} and then analyze the growth of the dimension of the spaces H^{0}(R̂_{g}, K⊗l ; R̂_{g}) of pluricanonical forms for all l ≥ 0. In this section we show that in doing so one only needs to consider forms defined on \overline{R}_{g} itself.

Theorem 6.1. We fix g ≥ 4 and let R̂_{g} → \overline{R}_{g} be any desingularisation. Then every pluricanonical form defined on the smooth locus \overline{R}_{g}^{reg} of \overline{R}_{g} extends holomorphically to R̂_{g}, that is, for all integers l ≥ 0 we have isomorphisms

H^{0}(\overline{R}_{g}^{reg}, K_{\overline{R}_{g}}^{⊗l}) ≅ H^{0}(R̂_{g}, K⊗l ; R̂_{g}).

A similar statement has been proved for the moduli space \overline{M}_{g} of curves cf. [HM] Theorem 1, and for the moduli space \overline{S}_{g} of spin curves, cf. [Lud] Theorem 4.1. We start by explicitly describing the locus of non-canonical singularities in \overline{R}_{g}, which has codimension 2. At a non-canonical singularity there exist local pluricanonical forms that do acquire poles on a desingularisation. We show that this situation does not occur for forms defined on the smooth locus \overline{R}_{g}^{reg}, and they extend holomorphically to R̂_{g}.

Definition 6.2. An automorphism of a Prym curve (X, η, β) is an automorphism σ ∈ Aut(X) such that there exists an isomorphism of sheaves γ : σ^{∗}η → η making the following diagram commutative.

\xymatrix{(σ^{∗}η)^{⊗2} \ar[r]^{γ^{⊗2}} \ar[d]_{σ^{∗}β} & η^{⊗2} \ar[d]^{β} \\ σ^{∗}O_{X} \ar[r]^{≃} & O_{X}}

If C := st(X) denotes the stable model of X then there is a group homomorphism Aut(X, η, β) → Aut(C) given by σ ↦ σ_{C}. The kernel Aut_{0}(X, η, β) of this homomorphism is called the subgroup of inessential automorphisms of (X, η, β).

Remark 6.3. The subgroup Aut_{0}(X, η, β) is isomorphic to {±1}^{CC(X̃)}/±1, where CC(X̃) is the set of connected components of the non-exceptional subcurve X̃ (compare [CCC] Lemma 2.3.2 and [Lud] Proposition 2.7). Given γ_{j} ∈ {±1} for every connected component X̃_{j} of X̃ consider the automorphism γ̃ of η̃ = η_{|X̃} which is multiplication by γ_{j} in

every fibre over X̃_{j}. Then there exists a unique inessential automorphism σ such that γ̃ extends to an isomorphism γ : σ^{∗}η → η compatible with the morphisms σ^{∗}β and β. Considering (−γ_{j})_{j} instead of (γ_{j})_{j} gives the same automorphism σ.

Definition 6.4. For a quasi-stable curve X, an irreducible component C_{j} is called an elliptic tail if p_{a}(C_{j}) = 1 and C_{j} ∩ \overline{(X − C_{j})} = \{p\}. The node p is then an elliptic tail node. A non-trivial automorphism σ of X is called an elliptic tail automorphism (with respect to C_{j}) if σ_{|X−C_{j}} is the identity.

Theorem 6.5. Let (X, η, β) be a Prym curve of genus g ≥ 4. The point [X, η, β] ∈ \overline{R}_{g} is smooth if and only if Aut(X, η, β) is generated by elliptic tail involutions.

Throughout this Appendix, X denotes a quasi-stable curve of genus g ≥ 2 and C := st(X) is its stable model. We denote by N ⊂ Sing(C) the set of exceptional nodes and Δ := Sing(C) − N. Then X is the support of a Prym curve if and only if N considered as a subgraph of the dual graph Γ(C) is eulerian, that is, every vertex of Γ(C) is incident to an even number of edges in N (cf. [BCF] Proposition 0.4).

Locally at a point [X, η, β], the moduli space \overline{R}_{g} is isomorphic to the quotient of the versal deformation space ℂ_{τ}^{3g−3} of (X, η, β) modulo the action of the automorphism group Aut(X, η, β). If ℂ_{t}^{3g−3} = Ext^{1}(Ω_{C}^{1}, O_{C}) denotes the versal deformation space of C, then the map ℂ_{τ}^{3g−3} → ℂ_{t}^{3g−3} is given by t_{i} = τ_{i}^{2} if (t_{i} = 0) ⊂ ℂ_{t}^{3g−3} is the locus where the exceptional node p_{i} ∈ N persists and t_{i} = τ_{i} otherwise. The morphism π : \overline{R}_{g} → \overline{M}_{g} is given locally by the map ℂ_{τ}^{3g−3}/ Aut(X, η, β) → ℂ_{t}^{3g−3}/Aut(C). One has the following decomposition of the versal deformation space of (X, η, β)

ℂ_{τ}^{3g−3} = ⨁_{p_{i}∈N} ℂ_{τ_{i}} ⊕ ⨁_{p_{i}∈Δ} ℂ_{τ_{i}} ⊕ ⨁_{C_{j}⊂C} H^{1}(C_{j}^{ν}, T_{C_{j}^{ν}}(−D_{j})),

where for a node p_{i} ∈ N we denote by (τ_{i} = 0) ⊂ ℂ_{τ}^{3g−3} the locus where the corresponding exceptional component E_{i} persists, while for a node p_{i} ∈ Δ we denote by (τ_{i} = 0) ⊂ ℂ_{τ}^{3g−3} the locus of those deformations in which p_{i} persists. Finally, for a component C_{j} ⊂ C with normalization C_{j}^{ν}, if D_{j} consists of the inverse images of the nodes of C under the normalization map C_{j}^{ν} → C_{j}, the group H^{1}(C_{j}^{ν}, T_{C_{j}^{ν}}(−D_{j})) parameterizes deformations of the pair (C_{j}^{ν}, D_{j}). This decomposition is compatible with the decomposition

ℂ_{t}^{3g−3} = ( ⨁_{p_{i}∈Sing(C)} ℂ_{t_{i}}) ⊕ (⨁_{C_{j}} H^{1}(C_{j}^{ν}, T_{C_{j}^{ν}}(−D_{j})))

as well as with the actions of the automorphism groups on ℂ_{τ}^{3g−3} and ℂ_{t}^{3g−3}, see also

[Lud] pg. 5. The point [X, η, β] ∈ \overline{R}_{g} is smooth if and only if the action of Aut(X, η, β) on ℂ_{τ}^{3g−3} is generated by quasi-reflections, that is, elements σ ∈ Aut(X, η, β) having 1 as an eigenvalue of multiplicity precisely 3g − 4. Theorem 6.5 follows from the following proposition.

Proposition 6.6. Let σ ∈ Aut(X, η, β) be an automorphism of a Prym curve (X, η, β) of genus g ≥ 4. Then σ acts on ℂ_{τ}^{3g−3} as a quasi-reflection if and only if X has an elliptic tail C_{j} such that σ is the elliptic tail involution with respect to C_{j}.

Proof. Let σ be an elliptic tail involution with respect to C_{j}. The induced automorphism σ_{C} is an elliptic tail involution of C and acts on the versal deformation space ℂ_{t}^{3g−3} of C as t_{1} ↦ −t_{1} and t_{i} ↦ t_{i}, i ≠ 1. Here t_{1} is the coordinate corresponding to the node p_{1} ∈ C_{j} ∩ \overline{(C − C_{j})}. The node p_{1} being non-exceptional, we have that t_{1} = τ_{1} hence σ · τ_{1} = −τ_{1}. If τ_{i} = t_{i}(i ≠ 1), then σ · τ_{i} = τ_{i}. For coordinates t_{i} = τ_{i}^{2}, σ is the identity in a neighbourhood of the corresponding exceptional component E_{i}, thus σ · τ_{i} = τ_{i}.

Now let σ ∈ Aut(X, η, β) act as a quasi-reflection with eigenvalues ζ and 1. As in the proof of [Lud] Proposition 2.15, there exists a node p_{1} ∈ C such that the action of σ is given by σ · τ_{1} = ζτ_{1} and σ · τ_{j} = τ_{j} for j ≠ 1. When p_{1} ∈ N, the induced automorphism σ_{C} acts via t_{1} ↦ ζ^{2}t_{1} and σ_{C} · t_{j} = t_{j} for j ≠ 1. If ζ^{2} ≠ 1, then σ_{C} acts as a quasi-reflection and p_{1} is an elliptic tail node, which contradicts the assumption p_{1} ∈ N. Therefore σ_{C} = Id_{C} and the exceptional component E_{1} over p_{1} is the only component on which σ acts non-trivially. The graph N ⊂ Γ(C) is eulerian and there exists a circuit of edges in N containing p_{1}.

[figure: C_{1}; C_{2}; C_{k} •; •; p_{k}; •; p_{1}; •; p_{2}; •; •; p_{i}; C_{3}; C_{i+1}; C_{i}]

By Remark 6.3, σ corresponds to an element ±(γ_{j})_{j} ∈ {±1}^{CC(X̃)}/ ± 1. Since σ acts non-trivially on E_{1} we find that γ_{1} = −γ_{2}. In particular, there exists i ≠ 1 such that σ acts non-trivially on E_{i}. This is a contradiction which shows that the node p_{1} is nonexceptional, τ_{1} = t_{1} and σ_{C} · t_{1} = ζt_{1} and σ_{C} · t_{i} = t_{i} for i ≠ 1. Thus σ_{C} is an elliptic tail involution of C with respect to an elliptic tail through the node p_{1} and ζ = −1. Since σ fixes every coordinate corresponding to an exceptional component of X, it follows that σ is an elliptic tail involution of X. □

Theorem 6.7. We fix g ≥ 4. A point [X, η, β] ∈ \overline{R}_{g} is a non-canonical singularity if and only if X has an elliptic tail C_{j} with j-invariant 0 and η is trivial on C_{j}.

The proof is similar to that of the analogous statement for \overline{S}_{g} and we refer to [Lud] Theorem 3.1 for a detailed outline of the proof and background on quotient singularities. Locally at [X, η, β], the space \overline{R}_{g} is isomorphic to a neighbourhood of the origin in ℂ_{τ}^{3g−3}/ Aut(X, η, β). We consider the normal subgroup H of Aut(X, η, β) generated by automorphisms acting as quasi-reflections on ℂ_{τ}^{3g−3}. The map ℂ_{τ}^{3g−3} → ℂ_{τ}^{3g−3}/H = ℂ_{υ}^{3g−3} is given by υ_{i} = τ_{i}^{2} if p_{i} is an elliptic tail node and υ_{i} = τ_{i} otherwise. The automorphism group Aut(X, η, β) acts on ℂ_{υ}^{3g−3} and the quotient ℂ_{υ}^{3g−3}/ Aut(X, η, β) is isomorphic to ℂ_{τ}^{3g−3}/ Aut(X, η, β). Since Aut(X, η, β) acts on ℂ_{υ}^{3g−3} without quasi-reflections the Reid–Shepherd-Barron–Tai criterion applies to this new action.

We fix an automorphism σ ∈ Aut(X, η, β) of order n and a primitive n-th root of unity ζ_{n}. If the action of σ on ℂ_{υ}^{3g−3} has eigenvalues ζ_{n}^{a_{1}}, …, ζ_{n}^{a_{3g−3}} with 0 ≤ a_{i} < n for i = 1, …, 3g − 3, then following [Re2] we define the age of σ by

age(σ, ζ_{n}) := \frac{1}{n} ∑_{i=1}^{n} a_{i}.

We say that σ satisfies the Reid–Shepherd-Barron–Tai inequality if age(σ, ζ_{n}) ≥ 1. The Reid–Shepherd-Barron–Tai criterion states that ℂ_{υ}^{3g−3}/ Aut(X, η, β) has canonical singularities if and only if every σ ∈ Aut(X, η, β) satisfies the Reid–Shepherd-Barron–Tai inequality (cf. [Re], [T],[Re2]).

Proof of the if-part of Theorem 6.7. Let (X, η, β) be a Prym curve, C = st(X) and C_{j} ⊂ X an elliptic tail with Aut(C_{j}) = ℤ_{6} and we assume η_{C_{j}} = O_{C_{j}}. We fix an elliptic tail automorphism σ_{C} with respect to C_{j} ⊂ C such that ord(σ_{C}) = 6. Then σ_{C} acts on ℂ_{t}^{3g−3} by t_{1} ↦ ζ_{6}t_{1}, t_{2} ↦ ζ_{6}^{2}t_{2} for an appropriate sixth root of unity ζ_{6} and σ ·t_{i} = t_{i} for i ≠ 1, 2. Here t_{1}, t_{2} ∈ Ext^{1}(Ω_{C}^{1}, O_{C}) correspond to smoothing the node p_{1} ∈ C_{j} ∩ \overline{(C − C_{j})} and deforming the curve [C_{j}, p_{1}] ∈ \overline{M}_{1,1} respectively. Since η_{C_{j}} = O_{C_{j}}, the automorphism σ_{C} lifts to an automorphism σ ∈ Aut(X, η, β) such that σ\overline{X−C_{j}} is the identity. Then σ

acts on ℂ_{τ}^{3g−3} as σ · τ_{1} = ζ_{6}τ_{1}, σ · τ_{2} = ζ_{6}^{2}τ_{2} and σ · τ_{i} = τ_{i} for i ≠ 1, 2. Since υ_{1} = τ_{1}^{2} and υ_{2} = τ_{2}, the action of σ on ℂ_{υ}^{3g−3} is υ_{1} ↦ ζ_{6}^{2}υ_{1}, υ_{2} ↦ ζ_{6}^{2}υ_{2} and υ_{i} ↦ υ_{i}, i ≠ 1, 2. We compute age(σ, ζ_{6}^{2}) = \frac{1}{3} + \frac{1}{3} +0+⋯+0 = \frac{2}{3} < 1, that is, [X, η, β] ∈ \overline{R}_{g} is a non-canonical singularity. Similarly, an elliptic tail automorphism of order 3 with respect to C_{j} acts via τ_{1} ↦ ζ_{3}^{2}τ_{1}, τ_{2} ↦ ζ_{3}τ_{2} and τ_{i} ↦ τ_{i}, i ≠ 1, 2, and then for the action on ℂ_{υ}^{3g−3} as υ_{1} ↦ ζ_{3}υ_{1}, υ_{2} ↦ ζ_{3}υ_{2} and υ_{i} ↦ υ_{i} for i ≠ 1, 2. This gives a value of \frac{2}{3} for the age. □

Suppose that [X, η, β] ∈ \overline{R}_{g} is a non-canonical singularity. Then there exists an automorphism σ ∈ Aut(X, η, β) of order n which acts on ℂ_{υ}^{3g−3} such that age(σ, ζ_{n}) < 1. Let p_{i_{0}}, p_{i_{1}} = σ_{C}(p_{i_{0}}), …, p_{i_{m−1}} = σ_{C}^{m−1}(p_{i_{0}}) be distinct nodes of C which are cyclicly permuted by the induced automorphism σ_{C} and p_{i_{j}} is not an elliptic tail node. The action of σ on the subspace ⨁_{j} ℂ_{τ_{i_{j}}} ⊂ ℂ_{τ}^{3g−3} is given by the matrix

B = (\begin{matrix}0 & c_{1} \\ ⋮ & & ... \\ 0 & & & c_{m−1} \\ c_{m} & 0 & ⋯ & 0 \end{matrix})

for appropriate scalars c_{j} ≠ 0. The pair ((X, η, β), σ) is said to be singularity reduced if for every such cycle we have that ∏_{j=1}^{m} c_{j} ≠ 1.

Proposition 6.8. ([HM], [Lud] Proposition 3.6) There exists a deformation (X′, η′, β′) of (X, η, β) such that σ deforms to an automorphism σ′ ∈ Aut(X′, η′, β′) and the nodes of every cycle of nodes as above with ∏_{j=1}^{m} c_{j} = 1 are smoothed. The pair ((X′, η′, β′), σ′) is then singularity reduced and the action of σ on ℂ_{υ}^{3g−3} and that of σ′ on ℂ_{υ′}^{3g−3} have the same eigenvalues and hence the same age.

We fix a singularity reduced pair ((X, η, β), σ) with n := ord(σ) ≥ 2 and assume that age(σ, ζ_{n}) < 1. We denote this assumption by (⋆). Using [Lud] Proposition 3.7 we obtain that if (⋆) holds, the induced automorphism σ_{C} of C = st(X) fixes every node with the possible exception of two nodes which are interchanged.

Proposition 6.9. If (⋆) holds, then σ_{C} fixes all components of the stable model C of X.

Proof. Let C_{i_{0}}, C_{i_{1}} = σ_{C}(C_{i_{0}}), …, C_{i_{m−1}} = σ_{C}^{m−1}(C_{i_{0}}) be distinct components of C, σ_{C}^{m}(C_{i_{0}}) = C_{i_{0}} and assume that m ≥ 2. Most of the proof of Proposition 3.8. in [Lud] applies to the case of Prym curves and implies that the normalization C_{i_{0}}^{ν} is rational and there are exactly three preimages of nodes p_{1}^{+}, p_{2}^{+}, p_{3}^{+} ∈ C_{i_{0}}^{ν} mapping to different nodes of C. By [Lud] Proposition 3.7 at least one of p_{1}, p_{2}, p_{3} is fixed by σ_{C}. If either one or all three nodes are fixed, then g(C) = 2, impossible. Thus two nodes, say p_{1} and p_{2}, are fixed by σ_{C} while p_{3} is interchanged with a fourth node p_{4}. Interchanging p_{3} and p_{4} gives a contribution of \frac{1}{2} to age(σ, ζ_{n}). Now consider the action of σ_{C} near p_{1} and let xy = 0 be a local equation of C at p_{1}. We have that t_{1} = xy ↦ yx = t_{1} and τ_{1} ↦ ±τ_{1}, where the minus sign is only possible if p_{1} ∈ N. Since p_{1} is not an elliptic tail node and ((X, η, β), σ) is singularity reduced, we have τ_{1} ↦ −τ_{1}, which gives an additional contribution of \frac{1}{2} to the age, that is, age(σ, ζ_{n}) ≥ \frac{1}{2} + \frac{1}{2} = 1, contradicting (⋆). □

Proposition 6.10 ([HM] p. 28, 36, [Lud] Proposition 3.9). We assume that (⋆) holds and denote by φ_{j} = σ_{|C_{j}^{ν}}^{ν} the induced automorphism of the normalization C_{j}^{ν} of the irreducible

component C_{j} of C. Then the pair (C_{j}^{ν}, φ_{j}) is one of the following types:

(i) φ_{j} = Id_{C_{j}^{ν}} and C_{j}^{ν} arbitrary. (ii) C_{j}^{ν} is rational and ord(φ_{j}) = 2, 4. (iii) C_{j}^{ν} is elliptic and ord(φ_{j}) = 2, 4, 3, 6. (iv) C_{j}^{ν} is hyperelliptic of genus 2 and φ_{j} is the hyperelliptic involution.

(v) C_{j}^{ν} is hyperelliptic of genus 3 and φ_{j} is the hyperelliptic involution.

(vi) C_{j}^{ν} is bielliptic of genus 2 and φ_{j} is the associated involution.

The possibility of σ_{C} interchanging two nodes does not appear, cf. [Lud] Prop. 3.10:

Proposition 6.11. Under the assumption (⋆), the automorphism σ_{C} fixes all the nodes of C.

Proposition 6.12. Assume (⋆) holds. Let C_{j} be a component of C with normalization C_{j}^{ν}, D_{j} the divisor of the marked points on C_{j}^{ν} and φ_{j} = σ_{|C_{j}^{ν}}^{ν}. Then (C_{j}^{ν}, D_{j}, φ_{j}) is of one of the

following types and the contribution to age(σ, ζ_{n}) coming from H^{1}(C_{j}^{ν}, T_{C_{j}^{ν}}(−D_{j})) ⊂ ℂ_{υ}^{3g−3} is at least the following quantity w_{j}:

(i) Identity component: φ_{j} = Id_{C_{j}^{ν}}, arbitrary pair (C_{j}^{ν}, D_{j}) and w_{j} = 0
(ii) Elliptic tail: C_{j}^{ν} is elliptic, D_{j} = p_{1}^{+} and p_{1}^{+} is fixed by φ_{j}.
order 2: ord(φ_{j}) = 2 and w_{j} = 0

order 4: C_{j}^{ν} has j-invariant 1728, ord(φ_{j}) = 4 and w_{j} = \frac{1}{2} order 3, 6: C_{j}^{ν} has j-invariant 0, ord(φ_{j}) = 3 or 6 and w_{j} = \frac{1}{3}

(iii) Elliptic ladder: C_{j}^{ν} is elliptic, D_{j} = p_{1}^{+} + p_{2}^{+}, with p_{1}^{+} and p_{2}^{+} both fixed by φ_{j}. order 2: ord(φ_{j}) = 2 and w_{j} = \frac{1}{2} order 4: C_{j}^{ν} has j-invariant 1728, ord(φ_{j}) = 4 and w_{j} = \frac{3}{4} order 3: C_{j}^{ν} has j-invariant 0, ord(φ_{j}) = 3 and w_{j} = \frac{2}{3}

(iv) Hyperelliptic tail: C_{j}^{ν} has genus 2, φ_{j} is the hyperelliptic involution, D_{j} is of the form D_{j} = p_{1}^{+} with p_{1}^{+} fixed by φ_{j} and w_{j} = \frac{1}{2}.

Proof. The proof is along the lines of the proof of Proposition 3.11 in [Lud]. The only difference occurs in the case of a singular elliptic tail on which σ acts with order 2. Assume that C_{j}^{ν} is rational, D_{j} = p_{1}^{+} + p_{1}^{−} + p_{2}, with ord(φ_{j}) = 2 which fixes p_{2}^{+} and interchanges p_{1}^{+} and p_{1}^{−}. If xy = 0 is an equation for C at p_{1}, then σ_{C} acts via t_{1} = xy ↦ yx = t_{1}. Since p_{1} is not an elliptic tail node and ((X, η, β), σ) is singularity reduced, the node p_{1} must be exceptional and σ · τ_{1} = −τ_{1}.

A deformation of (X, η, β) over the locus (τ_{i} = 0)_{i≠1} ⊂ ℂ_{τ}^{3g−3} smooths p_{1}. Furthermore, σ deforms to an automorphism σ′ of a general Prym curve (X′, η′, β′) over this locus, φ_{j} deforms to the involution φ′_{j} on the smooth elliptic tail C′_{j} such that it fixes the line bundle η′_{C′_{j}}, and the restrictions of σ and σ′ to the complement of C_{j} resp. C′_{j} co-

incide. Over the non-exceptional subcurve X̃ ⊂ X we have (σ′)^{∗}η′ ≅ η′. Thus σ ·τ_{1} = τ_{1} which is a contradiction. The case of a singular elliptic tail is thus excluded. □

Proposition 6.13. Under the hypothesis (⋆), the hyperelliptic tail case does not occur.

Proof. Let C_{j} be a genus 2 tail of C and C_{j′} the second component through p_{1}. The action of σ on H^{1}(C_{j}^{ν}, T_{C_{j}^{ν}}(−D_{j})) contributes \frac{1}{2} to the age of σ and C_{j′} has to be one of the cases of Proposition 6.12. If C_{j′} is elliptic, then g(C) = 3. If C_{j′} is a hyperelliptic tail or an elliptic ladder, the action on H^{1}(C_{j′}^{ν}, T_{C_{j′}^{ν}} (−D_{j′})) contributes at least \frac{1}{2}. Therefore C_{j′} is an identity component. If xy = 0 is an equation for C at p_{1}, then σ_{C} acts via t_{1} = xy ↦ −xy = −t_{1}. The node p_{1} is disconnecting, hence non-exceptional, and it is not an elliptic tail node. Therefore, υ_{1} = τ_{1} = t_{1} and σ acts as σ · υ_{1} = −υ_{1}. This gives an additional contribution of \frac{1}{2} to the age of σ finishing the proof. □

Proposition 6.14. In situation (⋆) the elliptic ladder cases do not occur.

Proof. Let C_{j} be an elliptic ladder of C of order n_{j} = ord(φ_{j}) and denote by C_{j′} resp. C_{j′′} the second component through the node p_{1} resp. p_{2}. Since every elliptic ladder contributes at least \frac{1}{2} to the age, C_{j′} and C_{j′′} can only be elliptic tails or identity components. If both are elliptic tails, then g(C) = 3, hence we may assume that C_{j′} is an identity component. If xy = 0 is an equation for C at p_{1}, then σ_{C} acts as x ↦ x, y ↦ αy and t_{1} ↦ αt_{1}, where α is a primitive n_{j}-th root of 1. If p_{1} is non-exceptional then υ_{1} = τ_{1} = t_{1} and the space H^{1}(C_{j}^{ν}, T_{C_{j}^{ν}}(−D_{j}))⊕ℂ·υ_{1} contributes to the age at least

1 = \begin{cases}\frac{1}{2} + \frac{1}{2} & if n_{j} = 2 \\ \frac{3}{4} + \frac{1}{4} & if n_{j} = 4 \\ \frac{2}{3} + \frac{1}{3} & if n_{j} = 3 \end{cases}

Therefore p_{1} ∈ N. Since N ⊂ Γ(C) is an eulerian subgraph, the node p_{2} is also exceptional, both p_{1} and p_{2} are non-disconnecting and C_{j′′} is an identity component as well. Moreover σ_{C} · t_{i} = αt_{i}, i = 1, 2. Since υ_{i} = τ_{i} and τ_{i}^{2} = t_{i} for i = 1, 2, we find that σ · υ_{i} = α_{i}υ_{i}, i = 1, 2, where α_{i} is a square root of α. Therefore, the contribution to the age of σ coming from H^{1}(C_{j}^{ν}, T_{C_{j}^{ν}}(−D_{j})) ⊕ ℂ · υ_{1} ⊕ ℂ · υ_{2} is at least

1 = \begin{cases}\frac{1}{2} + \frac{1}{4} + \frac{1}{4} & if n_{j} = 2 \\ \frac{3}{4} + \frac{1}{8} + \frac{1}{8} & if n_{j} = 4 \\ \frac{2}{3} + \frac{1}{6} + \frac{1}{6} & if n_{j} = 3 \end{cases}

and the case of elliptic ladders is excluded. □

Proposition 6.15. Under hypothesis (⋆), the case of an elliptic tail of order 4 does not occur.

Proof. Let C_{j} be an elliptic tail of order 4 and C_{j′} another component of C through p_{1}. Then σ_{C|C′_{j}} = Id_{C′_{j}} and σ_{C} acts as t_{1} = xy ↦ ζ_{4}xy = ζ_{4}t_{1} for a suitable fourth root ζ_{4} of 1. Since p_{1} is an elliptic tail node, we have υ_{1} = t_{1}^{2} and σ · v_{1} = −v_{1}. The action of σ on H^{1}(C_{j}^{ν}, T_{C_{j}^{ν}}(−D_{j})) ⊕ ℂ · υ_{1} contributes ≥ \frac{1}{2} + \frac{1}{2} = 1 to age(σ, ζ_{4}) excluding this case. □

Proposition 6.16. In situation (⋆) there has to be at least one elliptic tail of order 3 or 6.

Proof. Assume to the contrary that every component of C is either an identity component or an elliptic tail of order 2. The action of σ on every space H^{1}(C_{j}^{ν}, T_{C_{j}^{ν}}(−D_{j})) is trivial. If p_{1} is the node of an elliptic tail of order 2, then σ_{C} · t_{1} = −t_{1} and we have υ_{1} = τ_{1}^{2} = t_{1}^{2} and σ · υ_{1} = υ_{1}. In case p_{1} is non-exceptional but not an elliptic tail node, σ_{C} · t_{1} = t_{1}. Since υ_{1} = τ_{1} = t_{1}, we find that σ fixes υ_{1}. If p_{1} ∈ N, then σ_{C} · t_{1} = t_{1} and υ_{1}^{2} = τ_{1}^{2} = t_{1} and σ acts as υ_{1} ↦ ±υ_{1}. Since age(σ, ζ_{n}) < 1, there is exactly one node p_{1} such that σ · υ_{1} = −υ_{1}, that is, σ acts as quasi-reflection on ℂ_{υ}^{3g−3}, a contradiction. □

Proof of the only-if-part of Theorem 6.7. We proved, that if ((X, η, β), σ) is a singularity reduced pair and age(σ, ζ_{n}) < 1, where n = ord(σ), there exists an elliptic tail C_{j} ⊂ C with Aut(C_{j}) = ℤ_{6} such that ord(σ_{C_{j}}) ∈ {3, 6}. Since σ_{C_{j}}^{∗}(η_{C_{j}}) ≅ η_{C_{j}}, we find that η_{C_{j}} = O_{C_{j}}. Let ((X, η, β), σ) be a pair consisting of a Prym curve and an automorphism such that the age(σ, ζ_{n}) < 1. By Proposition 6.8 we may deform ((X, η, β), σ) to a singularity reduced pair ((X′, η′, β′), σ′) such that the actions of σ on ℂ_{υ}^{3g−3} and σ′ on ℂ_{υ′}^{3g−3} have the same ages. Therefore X′ has an elliptic tail C′_{j} with Aut(C′_{j}) = ℤ_{6} such that η′_{C′_{j}}

is trivial and σ′ acts on C′_{j} of order 3 or 6. In the deformation of (X, η, β) to (X′, η′, β′) elliptic tails are preserved hence ((X, η, β), σ) enjoys the same properties. □

Remark 6.17. If σ ∈ Aut(X, η, β) satisfies the inequality age(σ, ζ_{n}) < 1 (with respect to the action on ℂ_{υ}^{3g−3}), then σ is an elliptic tail automorphism and ord(σ) ∈ {3, 6}. Indeed, we already know that σ_{C} ∈ Aut(C) acts with order 3 or 6 on an elliptic tail C_{j}.

The action of σ on H^{1}(C_{j}^{ν}, T_{C_{j}^{ν}}(−D_{j})) and the υ-coordinate corresponding to the elliptic tail node on C_{j} contributes at least \frac{2}{3} to age(σ, ζ_{n}). Thus there is exactly one elliptic tail of order 3 or 6 and σ_{C} is an elliptic tail automorphism of the same order. If σ is not an elliptic tail automorphism of X, then there exists an exceptional component E_{1} ⊂ X on which σ acts non-trivially. Since E_{1} connects two non-exceptional components of X on which σ acts trivially, σ · υ_{1} = −υ_{1}, giving a contribution of \frac{1}{2} and an age ≥ \frac{2}{3} + \frac{1}{2} ≥ 1.

Proof of Theorem 6.1. We start with a pluricanonical form ω on \overline{R}_{g}^{reg} and show that ω lifts to a desingularization of a neighbourhood of every point [X, η, β] ∈ \overline{R}_{g}. We may assume that [X, η, β] is a general non-canonical singularity of \overline{R}_{g}, hence X = C_{1} ∪_{p} C_{2}, where [C_{1}, p] ∈ M_{g−1,1} is general and [C_{2}, p] ∈ M_{1,1} has j-invariant 0. Furthermore η_{C_{2}} = O_{C_{2}} and η_{1} := η_{C_{1}} ∈ Pic^{0}(C_{1})[2]. We consider the pencil ϕ : \overline{M}_{1,1} ⟶ \overline{R}_{g} given by ϕ[C′, p] = [C′ ∪_{p} C_{1}, η_{C′} = O_{C′}, η_{C_{1}} = η_{1}]. Since ϕ(\overline{M}_{1,1})∩Δ_{0}^{ram} = ∅, we imitate [HM] pg. 41-44 and construct an explicit open neighbourhood \overline{R}_{g} ⊃ S ⊃ ϕ(\overline{M}_{1,1}) such that the restriction to S of π : \overline{R}_{g} → \overline{M}_{g} is an isomorphism and every form ω ∈ H^{0}(\overline{R}_{g}^{reg}, K_{\overline{R}_{g}^{reg}}^{⊗l})

extends to a resolution Ŝ of S. For an arbitrary non-canonical singularity we show that ω extends locally to a desingularizaton along the lines of [Lud] Theorem 4.1. □

References

[ACGH] E. Arbarello, M. Cornalba, P. Griffiths and J. Harris, Geometry of algebraic curves, Grundlehren der mathematischen Wissenschaften 267, Springer.

[ACV] D. Abramovich, A. Corti and A. Vistoli, Twisted bundles and admissible coverings, math.AG/0106211, Comm. Algebra 31 (2003), 3547-3618.

[BCF] E. Ballico, C. Casagrande and C. Fontanari, Moduli of Prym curves, Documenta Mathematica 9 (2004), 265-281.

[B] A. Beauville, Prym varieties and the Schottky problem, Inventiones Math. 41 (1977), 149-96.

[Be] M. Bernstein, Moduli of curves with level structures, Harvard University Ph.D. Thesis, April 1999.

[CCC] L. Caporaso, C. Casagrande, and M. Cornalba, Moduli of roots of line bundles on curves, Transactions American Matematical Society 359 (2007), 3733–3768.

[Ca] F. Catanese, On the rationality of certain moduli spaces related to curves of genus 4, Springer Lecture Notes in Mathematics 1008 (1983), 30-50.

[C] M. Cornalba, A remark on the Picard group of spin moduli space, Rend. Mat. Accad. Lincei 2 (1991), 211-217.

[CH] M. Cornalba and J. Harris, Divisor classes associated to families of stable varieties, with applications to the moduli space of curves, Annales Scient. École Normale Supérieure 21 (1988), 455-475.

[CM] M. Coppens and G. Martens, Linear series on general k-gonal curves, Abh. Mathematisches Seminar Univ. Hamburg 69 (1999), 347-371.

[De] O. Debarre, Sur le Probleme de Torelli pour les Variétés des Prym, American Journal of Mathematics, 111 (1989), 111-134.

[Dol] I. Dolgachev, Rationality of fields of invariants, Algebraic Geometry Bowdoin 1985, Proceedings of Symphosia in Pure Mathematics, Volume 46-Part 2, 3-16.

[D1] R. Donagi, The unirationality of A_{5}, Annals of Mathematics 119 (1984), 269-307.

[D2] R. Donagi, The fibers of the Prym map, Contemporary Math. 136 (1992), 55-125, math.AG/9206008.

[DS] R. Donagi and R. Smith, The structure of the Prym map, Acta Mathematica 146 (1981), 25-102.

[EH1] D. Eisenbud and J. Harris, Limit linear series: basic theory, Inventiones Math. 85 (1986), 337-371.

[F1] G. Farkas, Syzygies of curves and the effective cone of \overline{M}_{g}, Duke Math. Journal 135 (2006), 53-98.

[F2] G. Farkas, Koszul divisors on moduli spaces of curves, math.AG/0607475, American Journal of Mathematics 131 (2009), to appear.

[F3] G. Farkas, Rational maps between moduli spaces of curves and Gieseker-Petri divisors, arXiv:0708.4188, Journal of Algebraic Geometry (2009), to appear.

[F4] G. Farkas, The Prym-Green Conjecture, in preparation.

[FMP] G. Farkas, M. Mustaţă and M. Popa, Divisors on M_{g,g+1} and the Minimal Resolution Conjecture for points on canonical curves, Annales Scientifique École Normale Supérieure 36 (2003), 553-581.

[FP] G. Farkas and M. Popa, Effective divisors on \overline{M}_{g}, curves on K3 surfaces and the Slope Conjecture, Journal of Algebraic Geometry 14 (2005), 151-174.

[FS] R. Friedman and R. Smith, The generic Torelli theorem for the Prym map, Inventiones Math. 67 (1982), 437-490.

[GL] M. Green and R. Lazarsfeld, Some results on the syzygies of finite sets and algebraic curves, Compositio Mathematica 67 (1988), 301-314.

[HF] H. Farkas, Unramified double coverings of hyperelliptic surfaces, J. Analyse Math. 30 (1976), 150-155.

[HM] J. Harris and D. Mumford, On the Kodaira dimension of \overline{M}_{g}, Inventiones Math. 67 (1982), 23-88.

[EH] D. Eisenbud and J. Harris, The Kodaira dimension of the moduli space of curves of genus ≥ 23, Inventiones Math. 90 (1987), 359-387.

[IGS] E. Izadi, M. Lo Giudice and G. Sankaran, The moduli space of étale double covers of genus 5 is unirational, arXiv:0802.0853.

[Kh] D. Khosla, Tautological classes on moduli spaces of curves with linear series and a push-forward formula when ρ = 0, arXiv:0704.1340.

[L] R. Lazarsfeld, A sampling of vector bundle techniques in the study of linear systems, in Lectures on Riemann Surfaces (Trieste 1987), World Scientific Press 1989, 500-559.

[Lud] K. Ludwig, On the geometry of the moduli space of spin curves, arXiv:0707.1831, Journal of Algebraic Geometry (2008), to appear.

[MM] S. Mori and S. Mukai, The uniruledness of the moduli space of curves of genus 11, Springer Lecture Notes in Mathematics 1016 (1983), 334353.

[R] M. Raynaud, Sections des fibrés vectoriels sur une courbe, Bull. Soc. Math. France 110 (1982), 103-125.

[Re] M. Reid, Canonical 3-folds, in Journées de Géometrie Algébrique d’Angers, Juillet 1979/Algebraic Geometry, Angers 1979, Sijthoff & Noordhoff, Alphen aan den Rijn (1980), 273-310.

[Re2] M. Reid, La correspondance de McKay, Séminaire Bourbaki, Vol. 1999/2000, Astérisque, 276 (2002), 53-72.

[T] Y. Tai, On the Kodaira dimension of the moduli space of abelian varieties, Inventiones Math. 68 (1982), 425-439.

[V] A. Verra, A short proof of the unirationality of A_{5}, Indagationes Math. 46 (1984), 339-355.

[V2] A. Verra, On the universal abelian variety of dimension 4, arXiv:0711.3890.

[Vo] C. Voisin, Green’s generic syzygy conjecture for curves of even genus lying on a K3 surface, Journal of European Math. Society 4 (2002), 363-404.

Humboldt-Universität zu Berlin, Institut Für Mathematik, 10099 Berlin

      E-mail address: farkas@math.hu-berlin.de

Leibniz Universität Hannover, Institut für Algebraische Geometrie D-30167 Hannover, Germany

      E-mail address: ludwig@math.uni-hannover.de