Packages

Contents

Introduction

For brevity, in this section we shall use the term function to include both functions and procedures.

The term intrinsic function or intrinsic refers to a function whose signature is stored in the system table of signatures. In terms of their origin, there are two kinds of intrinsics, system intrinsics (or standard functions) and user intrinsics, but they are indistinguishable in their use. A system intrinsic is an intrinsic that is part of the definition of the Magma system, whereas a user intrinsic is an informal addition to Magma, created by a user of the system. While most of the standard functions in Magma are implemented in C, a growing number are implemented in the Magma language. User intrinsics are defined in the Magma language using a package mechanism (the same syntax, in fact, as that used by developers to write standard functions in the Magma language).

This section explains the construction of user intrinsics by means of packages. From now on, intrinsic will be used as an abbreviation for user intrinsic.

It is useful to summarize the properties possessed by an intrinsic function that are not possessed by an ordinary user-defined function. Firstly, the signature of every intrinsic function is stored in the system's table of signatures. In particular, such functions will appear when signatures are listed and printing the function's name will produce a summary of the behaviour of the function. Secondly, intrinsic functions are compiled into the Magma internal pseudo-code. Thus, once an intrinsic function has been debugged, it does not have to be compiled every time it is needed. If the definition of the function involves a large body of code, this can save a significant amount of time when the function definition has to be loaded.

An intrinsic function is defined in a special type of file known as a package. In general terms a package is a Magma source file that defines constants, one or more intrinsic functions, and optionally, some ordinary functions. The definition of an intrinsic function may involve Magma standard functions, functions imported from other packages and functions whose definition is part of the package. It should be noted that constants and functions (other than intrinsic functions) defined in a package will not be visible outside the package, unless they are explicitly imported.

The syntax for the definition of an intrinsic function is similar to that of an ordinary function except that the function header must define the function's signature together with text summarizing the semantics of the function. As noted above, an intrinsic function definition must reside in a package file. It is necessary for Magma to know the location of all necessary package files. A package may be attached or detached through use of the Attach or Detach procedures. More generally, a family of packages residing in a directory tree may be specified through provision of a spec file which specifies the locations of a collection of packages relative to the position of the spec file. Automatic attaching of the packages in a spec file may be set by means of an environment variable (MAGMA_SYSTEM_SPEC for the Magma system packages and MAGMA_USER_SPEC for a users personal packages).

So that the user does not have to worry about explicitly compiling packages, Magma has an auto-compile facility that will automatically recompile and reload any package that has been modified since the last compilation. It does this by comparing the time stamp on the source file (as specified in an Attach procedure call or spec file) with the time stamp on the compiled code. To avoid the possible inefficiency caused by Magma checking whether the file is up to date every time an intrinsic function is referenced, the user can indicate that the package is stable by including the freeze; directive at the top of the package containing the function definition.

A constant value or function defined in the body of a package may be accessed in a context outside of its package through use of the import statement. The arguments for an intrinsic function may be checked through use of the require statement and its variants. These statements have the effect of generating an error message at the level of the caller rather than in the called intrinsic function.

See also the section on user-defined attributes for the declare attributes directive to declare user-defined attributes used by the package and related packages.

Intrinsics

Besides the definition of constants at the top, a package file just consists of intrinsics. There is only one way a intrinsic can be referred to (whether from within or without the package). When a package is attached, its intrinsics are incorporated into Magma. Thus intrinsics are `global' --- they affect the global Magma state and there is only one set of Magma intrinsics at any time. There are no `local' intrinsics.

A package may contain undefined references to identifiers. These are presumed to be intrinsics from other packages which will be attached subsequent to the loading of this package.

intrinsic name(arguments) [ -> return-values ]
{ comment text }
      statements
end intrinsic;
The syntax of a intrinsic declaration is as above, where: name is the name of the intrinsic, which can be any identifier or a non-alphanumeric name enclosed within single quotes (like '+'); arguments is the argument list (possibly variadic and optionally including parameters preceded by a colon); optionally there is an arrow and return type list return-values; the comment text is any text within the braces (precede } by a backslash to get a right brace within the text, and use " to repeat the comment from the immediately preceding intrinsic); and statements is a list of statements making up the body of the intrinsic.

arguments is a list of comma-separated arguments of the form

 name::type
~name::type
~name

where name is the name of the argument (any identifier), and type designates the type, which can be either a simple category name, an extended type, or one of the following:

.                  Any type
<category-name>    Simple type name
[]                 Sequence type
{}                 Set type
{[ ]}              Set or sequence type
{@ @}              Iset type
{* *}              Multiset type
<>                 Tuple type

or a composite type:

[ <type> ]    Sequences over <type>
{ <type> }    Sets over <type>
{[ <type> ]}  Sets or sequences over <type>
{@ <type> @}  Indexed sets over <type>
{* <type> *}  Multisets over <type>

where type is either a simple or extended type. The reference form ~name::type requires that the input argument must be initialized to an object of that type. The reference form ~name is a plain reference argument --- it need not be initialized. Parameters may also be specified --- these are just as in functions and procedures (preceded by a colon). If the final non-parameter entry in arguments is " ... " then the intrinsic is variadic, with semantics similar to that of a variadic function, described previously.

return-values is a list of comma-separated simple types. If there is an arrow and the return list, the intrinsic is assumed to be functional; otherwise it is assumed to be procedural.

The body of statements should return the correct number and types of arguments if the intrinsic is functional, while the body should return nothing if the intrinsic is procedural.

Example Func_intrinsic (H2E7)

A functional intrinsic for greatest common divisors taking two integers and returning another:
   intrinsic myGCD(x::RngIntElt, y::RngIntElt) -> RngIntElt
   { Return the GCD of x and y }
      return ...;
   end intrinsic;
A procedural intrinsic for Append taking a reference to a sequence Q and any object then modifying Q:
   intrinsic Append(~Q::SeqEnum, . x)
   { Append x to Q }
       ...;
   end intrinsic;
A functional intrinsic taking a sequence of sets as arguments 2 and 3:
   intrinsic IsConjugate(G::GrpPerm, R::[ { } ], S::[ { } ]) -> BoolElt
   { True iff partitions R and S of the support of G are conjugate in G }
       return ...;
   end intrinsic;

Resolving Calls to Intrinsics

It is often the case that many intrinsics share the same name. For instance, the intrinsic Factorization has many implementations for various object types. We will call such intrinsics overloaded intrinsics, or refer to each of the participating intrinsics as an overload. When the user calls such an overloaded intrinsic, Magma must choose the "best possible" overload.

Magma's overload resolution process is quite simple. Suppose the user is calling an intrinsic of arity r, with a list of parameters < p1, ..., pr >. Let the tuple of the types of these parameters be < t1, ..., tr >, and let S be the set of all relevant overloads (that is, overloads with the appropriate name and of arity r). We will represent overloads as r-tuples of types.

To pick the "best possible" overload, for each parameter p ∈{ p1, ..., pr }, Magma finds the set Si ⊆S of participating intrinsics which are the best matches for that parameter. More specifically, an intrinsic s = < u1, ..., ur > is included in Si if and only if ti is a ui, and no participating intrinsic s' = < v1, ..., vr > exists such that ti is a vi and vi is a ui. Once the sets Si are computed, Magma finds their intersection. If this intersection is empty, then there is no match. If this intersection has cardinality greater than one, then the match is ambiguous. Otherwise, Magma calls the overload thus obtained.

An example at this point will make the above process clearer:

Example Func_intrinsic-lookup (H2E8)

We demonstrate Magma's lookup mechanism with the following example. Suppose we have the following overloaded intrinsics:
   intrinsic overloaded(x::RngUPolElt, y::RngUPolElt) -> RngIntElt
   { Overload 1 }
      return 1;
   end intrinsic;
   intrinsic overloaded(x::RngUPolElt[RngInt], y::RngUPolElt) -> RngIntElt
   { Overload 2 }
      return 2;
   end intrinsic;
   intrinsic overloaded(x::RngUPolElt, y::RngUPolElt[RngInt]) -> RngIntElt
   { Overload 3 }
      return 3;
   end intrinsic;
   intrinsic overloaded(x::RngUPolElt[RngInt], y::RngUPolElt[RngInt]) -> RngIntElt
   { Overload 4 }
      return 4;
   end intrinsic;

The following Magma session illustrates how the lookup mechanism operates for the intrinsic overloaded:

> R1<x> := PolynomialRing(Integers());
> R2<y> := PolynomialRing(Rationals());
> f1 := x + 1;
> f2 := y + 1;
> overloaded(f2, f2);
1
> overloaded(f1, f2);
2
> overloaded(f2, f1);
3
> overloaded(f1, f1);
4

Attaching and Detaching Package Files

The procedures Attach and Detach are provided to attach or detach package files. Once a file is attached, all intrinsics within it are included in Magma. If the file is modified, it is automatically recompiled just after the user hits return and just before the next statement is executed. So there is no need to re-attach the file (or `re-load' it). If the recompilation of a package file fails (syntax errors, etc.), all of the intrinsics of the package file are removed from the Magma session and none of the intrinsics of the package file are included again until the package file is successfully recompiled. When errors occur during compilation of a package, the appropriate messages are printed with the string `[PC]' at the beginning of the line, indicating that the errors are detected by the Magma package compiler.

If a package file contains the single directive freeze; at the top then the package file becomes frozen --- it will not be automatically recompiled after each statement is entered into Magma. A frozen package is recompiled if need be, however, when it is attached (thus allowing fixes to be updated) --- the main point of freezing a package which is `stable' is to stop Magma looking at it between every statement entered into Magma interactively.

When a package file is complete and tested, it is usually installed in a spec file so it is automatically attached when the spec file is attached. Thus Attach and Detach are generally only used when one is developing a single package file containing new intrinsics.

Attach(F) : MonStgElt ->
Procedure to attach the package file F.
Detach(F) : MonStgElt ->
Procedure to detach the package file F.
freeze;
Freeze the package file in which this appears at the top.

Related Files

There are two files related to any package source file file.m:

file.sig     sig file containing signature information;
file.lck     lock file.

The lock file exists while a package file is being compiled. If someone else tries to compile the file, it will just sit there till the lock file disappears. In various circumstances (system down, Magma crash) .lck files may be left around; this will mean that the next time Magma attempts to compile the associated source file it will just sit there indefinitely waiting for the .lck file to disappear. In this case the user should search for .lck files that should be removed.

Importing Constants

import "filename": ident_list;
This is the general form of the import statement, where "filename" is a string and ident_list is a list of identifiers.

The import statement is a normal statement and can in fact be used anywhere in Magma, but it is recommended that it only be used to import common constants and functions/procedures shared between a collection of package files. It has the following semantics: for each identifier I in the list ident_list, that identifier is declared just like a normal identifier within Magma. Within the package file referenced by filename, there should be an assignment of the same identifier I to some object O. When the identifier I is then used as an expression after the import statement, the value yielded is the object O.

The file that is named in the import statement must already have been attached by the time the identifiers are needed. The best way to achieve this in practice is to place this file in the spec file, along with the package files, so that all the files can be attached together.

Thus the only way objects (whether they be normal objects, procedures or functions) assigned within packages can be referenced from outside the package is by an explicit import with the `import' statement.

Example Func_import (H2E9)

Suppose we have a spec file that lists several package files. Included in the spec file is the file defs.m containing:
       MY_LIMIT := 10000;
       function fred(x)
   	    return 1/x;
       end function;
Then other package files (in the same directory) listed in the spec file which wish to use these definitions would have the line
	import "defs.m": MY_LIMIT, fred;
at the top. These could then be used inside any intrinsics of such package files. (If the package files are not in the same directory, the pathname of defs.m will have to be given appropriately in the import statement.)

Argument Checking

Using `require' etc. one can do argument checking easily within intrinsics. If a necessary condition on the argument fails to hold, then the relevant error message is printed and the error pointer refers to the caller of the intrinsic. This feature allows user-defined intrinsics to treat errors in actual arguments in exactly the same way as they are treated by the Magma standard functions.

require condition: print_args;
The expression condition may be any yielding a Boolean value. If the value is false, then print_args is printed and execution aborts with the error pointer pointing to the caller. The print arguments print_args can consist of any expressions (depending on arguments or variables already defined in the intrinsic).
requirerange v, L, U;
The argument variable v must be the name of one of the argument variables (including parameters) and must be of integer type. The bounds L and U may be any expressions each yielding an integer value. If v is not in the range [L, ..., U], then an appropriate error message is printed and execution aborts with the error pointer pointing to the caller.
requirege v, L;
The argument variable v must be the name of one of the argument variables (including parameters) and must be of integer type. The bound L must yield an integer value. If v is not greater than or equal to L, then an appropriate error message is printed and execution aborts with the error pointer pointing to the caller.

Example Func_require (H2E10)

A trivial version of Binomial(n, k) which checks that n≥0 and 0 ≤k ≤n.

   intrinsic Binomial(n::RngIntElt, k::RngIntElt) -> RngIntElt
   { Return n choose k }
       requirege n, 0;
       requirerange k, 0, n;
       return Factorial(n) div Factorial(n - k) div Factorial(k);
   end intrinsic;
A simple function to find a random p-element of a group G.
   intrinsic pElement(G::Grp, p::RngIntElt) -> GrpElt
   { Return p-element of group G }
      require IsPrime(p): "Argument 2 is not prime";
      x := random{x: x in G | Order(x) mod p eq 0};
      return x^(Order(x) div p);
   end intrinsic;

The Number of Results

Since V2.28, a function intrinsic can tell how many results are requested by the caller of the function. This means the intrinsic can possibly be more efficient in the case that less results are required than the default full list of results.

NumberOfResults() : -> RngIntElt, [ BoolElt ]
Nresults() : -> RngIntElt, [ BoolElt ]
Within a function or intrinsic function, return the number n of results requested by the caller of the current function, and optionally return a sequence B of booleans indicating which specific results are requested. If the function is called in the context of printing, then n is zero (and B is empty).

Example Func_Nresults (H2E11)

Suppose that the file "test.m" contains the following intrinsic function.

   intrinsic Func(x::RngIntElt) -> ., .
   {Test function}
     n := Nresults();
     printf "[Nresults:
     if n eq 1 then
       return x;
     else // includes print mode
       // Could potentially do something very expensive
       return x, x^2;
     end if;
   end intrinsic;
The intrinsic function can thus be called:
> Attach("test.m");
> Func(3); // print (returns all by default)
[Nresults: 0]
3 9
> c := Func(5); // Avoid potentially expensive computation of 2nd result
[Nresults: 1]
> c;
5
> a, b := Func(4);
[Nresults: 2]
> a, b;
4 16
Now suppose that the file "test.m" also contains the following intrinsic function. The function simulates an algorithm to compute the Smith form S of the input matrix A, with optional unimodular transformation matrices P and Q so that S=P x A x Q (here we return strings to simulate the actual matrices which would be computed). The algorithm only computes P and/or Q if the caller requests these, based on the LHS of the assignment statement (the internal intrinsic function SmithForm has similar optimisations).
   intrinsic MySmith(A::Mtrx) -> ., .
   {Test function}
     n, B := Nresults();
     printf "[Nresults:
     if n le 1 then
       // Compute Smith S only (includes print mode)
       return "S", _, _;
     elif n eq 2 then
       // Compute Smith S and P so S = P*X*?
       return "S", "P", _;
     else // n eq 3
       if not B[2] then
         // Compute Smith S and Q so S = ?*X*Q
         return "S", _, "Q"; // S = ?*X*Q
       else
         // Compute Smith S and P, Q so S = P*X*Q
         return "S", "P", "Q"; // S = P*X*Q
       end if;
     end if;
   end intrinsic;
After attaching the file, the intrinsic function can thus be called:
> A := MatrixRing(IntegerRing(), 2)!1; // arbitrary matrix
> MySmith(A); // print mode
[Nresults: 0, B: []]
S
> S := MySmith(A);
[Nresults: 1, B: [ true ]]
> S, P := MySmith(A);
[Nresults: 2, B: [ true, true ]]
> S, _, Q := MySmith(A);
[Nresults: 3, B: [ true, false, true ]]
> S, P, Q := MySmith(A);
[Nresults: 3, B: [ true, true, true ]]

Package Specification Files

A spec file (short for `specification file') lists a complete tree of Magma package files. This makes it easy to collect many package files together and attach them simultaneously.

The specification file consists of a list of tokens which are just space-separated words. The tokens describe a list of package files and directories containing other packages. The list is described as follows. The files that are to be attached in the directory indicated by S are listed enclosed in { and } characters. A directory may be listed there as well, if it is followed by a list of files from that directory (enclosed in braces again); arbitrary nesting is allowed this way. A filename of the form +spec is interpreted as another specification file whose contents will be recursively attached when AttachSpec (below) is called. The files are taken relative to the directory that contains the specification file. See also the example below.

AttachSpec(S) : MonStgElt ->
If S is a string indicating the name of a spec file, this command attaches all the files listed in S. The format of the spec file is given above.
DetachSpec(S) : MonStgElt ->
If S is a string indicating the name of a spec file, this command detaches all the files listed in S. The format of the spec file is given above.

Example Func_spec (H2E12)

Suppose we have a spec file /home/user/spec consisting of the following lines:
{
   Group
   {
      chiefseries.m
      socle.m
   }
   Ring
   {
      funcs.m
      Field
      {
            galois.m
      }
   }
}
Then there should be the files
      /home/user/spec/Group/chiefseries.m
      /home/user/spec/Group/socle.m
      /home/user/spec/Ring/funcs.m
      /home/user/spec/Ring/Field/galois.m
and if one typed within Magma
      AttachSpec("/home/user/spec");
then each of the above files would be attached. If instead of the filename galois.m we have +galspec, then the file /home/user/spec/Ring/Field/galspec would be a specification file itself whose contents would be recursively attached.

User Startup Specification Files

The user may specify a list of spec files to be attached automatically when Magma starts up. This is done by setting the environment variable MAGMA_USER_SPEC to a colon separated list of spec files.

Example Func_startup-spec (H2E13)

One could have
      setenv MAGMA_USER_SPEC "$HOME/Magma/spec:/home/friend/Magma/spec"
in one's .cshrc . Then when Magma starts up, it will attach all packages listed in the spec files $HOME/Magma/spec and /home/friend/Magma/spec.
V2.28, 13 July 2023