# Module Topkg.Pkg

    module Pkg: sig .. end

Package description.

See the basics.

## Installation description

The installation description generates an opam install file which is simply a description of file moves (in the `mv` sense) from the build or source directory to standard install directories. Describing these moves in a given build configuration effectively determines what needs to built by the package build command.

Note. Always use `"/"` as a directory seperator for paths, even on Windows.

    type install

The type for representing a set of install moves.

    val nothing : install

`nothing` is an empty set of install moves.

    val flatten : install list -> install

`flatten installs` is the union of all the install moves in `installs`.

    type field = ?force:bool ->
           ?built:bool ->
           ?cond:bool ->
           ?exts:Topkg.Exts.t -> ?dst:Topkg.fpath -> Topkg.fpath -> install

The type for an install field, a function that describe file moves to a particular installation directory. In the simplest form a call `field src` simply moves the file `src` at the root of the install directory of the field.

In general `field ~force ~built ~cond ~exts ~dst src` generates install move as follows:

- `dst` is the path were the source is written to. Expressed relative to the install directory of the field. Defaults to `Fpath.basename src`, i.e. at the root of the install directory. If `dst` is a directory path, the destination is `(dst ^ Fpath.basename src)`.
- If `exts` is present and non empty, generates the list of paths `List.map (fun e -> src ^ e)` and a move for each of these. For example `field ~exts:``Topkg.Exts.api``"src/m"` would generate a move for the files `"src/m.mli"`, `"src/m.cmi"`, `"src/m.cmti"`, `"src/m.cmx"`.
- If `cond` is `false` (defaults to `true`) no file move is generated. This provides a convenient way to conditionalize installation based on the build configuration for example:

      let jsoo = Conf.value jsoo in
      Pkg.mllib ~cond:jsoo "src/mylib_jsoo.mllib"

- If `built` is `true` (default), `src` is expressed relative to the build directory of the distribution and the path `src` is be given to build system invocation for construction. If `false`, `src` is relative to the root of the distribution and is excluded from the build system invocation; this can be used for installing files that don't need to be built.
- If `force` is `true` (defaults to `false`) it disables the automatic `src` filtering performed by the library. When `false`, the library automatically disable certain build artefact depending on OCaml's configuration, one such example is filtering native code build artefact if the OCaml install doesn't support native code compilation

    type exec_field = ?auto:bool -> field

The type for fields that install executable files. This is like `Topkg.Pkg.field` except for the additional `auto` parameter:

- If `auto` is `true` (default) then `field src dst` automatically adds the `".native"` suffix to `src` if `Topkg.Conf.OCaml.native` is `true` and the `".byte"` suffix otherwise. Besides it automatically adds `Topkg.Exts.exe` to `dst`, which handles things correctly on the various Windows ports.
- If `auto` is `false` you have full control according to `Topkg.Pkg.field`.

    val bin : exec_field

`bin` is a field that installs to a common `bin/` directory.

    val doc : field

`doc` is a field that installs to a package specific `doc/` directory

    val etc : field

`etc` is a field that installs to a package specific `etc/` directory.

    val lib : field

`lib` is a field that installs to a package specific `lib/` directory.

    val lib_root : field

`lib_root` is a field that install to a common `lib/` directory.

    val libexec : exec_field

`libexec` is a field that installs to a package specific `lib/` directory but with the executable bit set.

    val libexec_root : exec_field

`libexec_root` is a field that install to a common `lib/` directory but with the executable bit set.

    val man : field

`man` is a field that installs to a common `man/` directory. See the opam manual for details.

    val misc : field

`misc` is a field that installs to an arbitrary absolute path, the user is prompted for authorization, see the opam manual for details.

    val sbin : exec_field

`sbin` is a field that installs to a common `sbin/` directory.

    val share : field

`share` is a field that installs to a package specific `share/` directory.

    val share_root : field

`share_root` is a field that installs to a common `share/` directory.

    val stublibs : field

`stublibs` is a field that install to a common `lib/stublibs/` directory. Used for OCaml C stub directory.

    val toplevel : field

`toplevel` is a field that installs to a common `lib/toplevel/` directory.

### Test executable description

    val test : ?run:bool -> ?dir:Topkg.fpath -> ?args:Topkg.Cmd.t -> exec_field

`test` is a special executable field: it doesn't install the described executable (as such the `dst` argument is ignored at the moment). If `run` is `true` (default) executes the test with `args` when `ocaml pkg/pkg.ml test` is run; this will notably run to test the distribution tarball. If `run` is `false` the test needs to be invoked explicitely. `dir` specifies the current working directory for the test, expressed relative to the root directory of the package (defaults to `.`).

### OCamlbuild higher-level installs

The following functions are OCamlbuild specific higher level installs that generate moves by reading OCamlbuild specification files. They also automatically handle the `Topkg.Conf.debugger_support` configuration key by building and installing the build artefacts needed by debuggers whenever its value is `true`.

    val mllib : ?field:field ->
           ?cond:bool ->
           ?cma:bool ->
           ?cmxa:bool ->
           ?cmxs:bool ->
           ?api:string list -> ?dst_dir:Topkg.fpath -> Topkg.fpath -> install

`mllib ~field ~cond ~cma ~cmxa ~cmxs ~api ~dst_dir mllib` installs an OCaml library described by the OCamlbuild .mllib file `mllib` with:

- `field` is the field where it gets installed (defaults to `Topkg.Pkg.lib`).
- If `cond` is `false` (defaults to `true`), no move is generated.
- `cma`, `cmxa`, `cmxs` determine if corresponding archives are built and installed, they all default to `true`.
- `api` is the list of modules that defines the public interface of the library, if `None` all the modules mentioned in `mllib` are part of the public interface.
- `dst_dir` is the destination directory of the library in the field. If unspecified this is the root of the field's directory.

    val clib : ?dllfield:field ->
           ?libfield:field ->
           ?cond:bool -> ?lib_dst_dir:Topkg.fpath -> Topkg.fpath -> install

`clib clib` installs C stubs described by the OCamlbuild .clib file `clib` with:

- `dllfield` is the field where the C DLL archive gets installed, (defaults to `Topkg.Pkg.stublibs`)
- `libfield` is the field where the C static archive gets installed (defaults to `Topkg.Pkg.lib`)
- If `cond` is `false` (defaults to `true`), no move is generated.
- `lib_dst_dir` is the destination directory of the library in the `libfield` field. If unspecified this is the root of the field's directory. This does not affect the `dllfield`, DLLs are always installed at the root directory of the `dllfield`.

## Build description

    type build

The type for package build description.

    val build : ?prepare_on_pin:bool ->
           ?dir:Topkg.fpath ->
           ?pre:(Topkg.Conf.t -> unit Topkg.result) ->
           ?cmd:(Topkg.Conf.t -> Topkg.Conf.os -> Topkg.fpath list -> unit Topkg.result) ->
           ?post:(Topkg.Conf.t -> unit Topkg.result) ->
           ?clean:(Topkg.Conf.os -> build_dir:Topkg.fpath -> unit Topkg.result) ->
           unit -> build

`build ~prepare_on_pin ~dir ~cmd ~pre ~post` describes the package build procedure.

- `prepare_on_pin` if `true` (default) distribution preparation is performed if a ``Pin` build context is detected. This means that the checkout is watermarked and the massage hook is invoked, see step 2. of distribution creation.
- `dir` is the directory where build artefacts are generated, (defaults to `"_build"`). Note that his value can be overriden from the command line.
- `pre` is a hook that is invoked with the build context, after distribution preparation if applicable, but before the build command. It can be used to adapt the build setup according to the build configuration. Default is a nop.
- `cmd` invokes the build system to build the files determined by install moves. It is given the build configuration, an OS specification, the list of files to build relative to the build directory, and build the given files in the build directory. The default is:

      fun c os files -> OS.Cmd.run @@ Cmd.(Pkg.build_cmd c os %% of_list files)

- `post` is a hook that is invoked with the build context after the build command returned sucessfully. Default is a nop.
- `clean` is invoked to clean a build. It is given an OS specification and a build directory to clean. The default is:

      let clean os ~build_dir = OS.Cmd.run @@ Pkg.clean_cmd os ~build_dir

Warning. If you are invoking tools in your hooks consider using `Topkg.Conf.tool` to look them up it helps for cross-compilation.

### OCamlbuild support

If you are using `ocamlbuild`, the following functions help to customize the build system invocation according to the configuration.

    val build_cmd : Topkg.Conf.t -> Topkg.Conf.os -> Topkg.Cmd.t

`build_cmd c os` is the default build command to which files to build are given. Its value is defined by:

    fun c os ->
      let ocamlbuild = Conf.tool "ocamlbuild" os in
      let build_dir = Conf.build_dir c in
      let toolchain =
        match Topkg_conf.toolchain c with
        | Some toolchain -> Topkg_cmd.(v "-toolchain" % toolchain)
        | _ -> Topkg_cmd.empty
      in
      let debug = Cmd.(on (Conf.debug c) (v "-tag" % "debug")) in
      let profile = Cmd.(on (Conf.profile c) (v "-tag" % "profile")) in
      Cmd.(ocamlbuild % "-use-ocamlfind" %% toolchain % "-classic-display" %%
                        debug %% profile % "-build-dir" % build_dir)

    val clean_cmd : Topkg.Conf.os -> build_dir:Topkg.fpath -> Topkg.Cmd.t

`clean_cmd os ~build_dir` is the default clean command. Its value is defined by:

    fun os ~build_dir ->
      let ocamlbuild = Conf.tool "ocamlbuild" os in
      Cmd.(ocamlbuild % "-use-ocamlfind" % "-classic-display" %
                        "-build-dir" % build_dir % "-clean")

    val ocb_tag : Topkg.Conf.t -> 'a Topkg.Conf.key -> string -> Topkg.Cmd.t

`ocb_tag c key name` is a command fragment adding the `ocamlbuild` parameterized tag `name` with `key`'s value to the default set of tags. The key value is converted to a string using the printer of the key value converter.

For example with a key `k : bool Conf.key` whose value is `true`, `ocb_tag c k "foo"` adds the tag `foo(true)` to the default tags. A sample build command for `Topkg.Pkg.build` using this key would be:

      let cmd c os files =
        OS.Cmd.run Cmd.(build_cmd c os %% ocb_tag c k "foo" %% of_list files)

    val ocb_bool_tag : Topkg.Conf.t -> bool Topkg.Conf.key -> string -> Topkg.Cmd.t

`ocb_bool_tag c key name` is a command fragment adding the `ocamlbuild` tag `name` to the default set of tags iff `key`'s value is `true`.

    val ocb_bool_tags : Topkg.Conf.t -> (bool Topkg.Conf.key * string) list -> Topkg.Cmd.t

`ocb_bool_tags c assoc` is the concatenation of `Topkg.Pkg.ocb_bool_tag` for all pairs in `assoc`.

## Distribution description

    type watermark = string *
           [ `Name
           | `Opam of Topkg.fpath option * string * string
           | `String of string
           | `Vcs of [ `Commit_id ]
           | `Version
           | `Version_num ]

The type for watermarks. A watermark identifier, e.g. `"ID"` and its definition:

- ``String s`, `s` is the definition.
- ``Name`, is the name of package.
- ``Version`, is the version of the distribution.
- ``Version_num`, is the version of the distribution with a potential leading `'v'` or `'V'` dropped.
- ``Vcs `Commit_id`, is the commit identifier (hash) of the distribution. May be post-fixed by `"dirty"` in dev package (``Pin`) builds.
- ``Opam (file, field, sep)`, is the values of the field `field` concatenated with separator `sep` of the opam file `file`, expressed relative to the distribution root directory, if `file` is `None` this is the package's default opam file, see `Topkg.Pkg.describe`. Not all fields are supported see the value of `Topkg_care.Opam.File.field_names`. Warning. In dev package (``Pin`) builds, ``Opam` watermarks are only substituted if the package `topkg-care` is installed.

When a file is watermarked with an identifier `"ID"`, any occurence of the sequence `%%ID%%` in its content is substituted by its definition.

    type distrib

The type for describing distribution creation.

    val distrib : ?watermarks:watermark list ->
           ?files_to_watermark:(unit -> Topkg.fpath list Topkg.result) ->
           ?massage:(unit -> unit Topkg.result) ->
           ?exclude_paths:(unit -> Topkg.fpath list Topkg.result) ->
           ?uri:string -> unit -> distrib

`distrib ~watermarks ~files_to_watermark ~massage
~exclude_paths ~uri ()` influences the distribution creation process performed by the `topkg` tool. See the full details about distribution creation.

In the following the distribution build directory is a private clone of the package's source repository's `HEAD` when `topkg distrib` is invoked.

- `watermarks` defines the source watermarks for the distribution, defaults to `Topkg.Pkg.watermarks`.
- `files_to_watermark` is invoked in the distribution build directory to determine the files to watermark, defaults to `Topkg.Pkg.files_to_watermark`.
- `massage` is invoked in the distribution build directory, after watermarking, but before archiving. It can be used to generate distribution time build artefacts. Defaults to `Topkg.Pkg.massage`.
- `exclude_paths ()` is invoked in the distribution build directory, after massaging, to determine the paths that are excluded from being added to the distribution archive. Defaults to `Topkg.Pkg.exclude_paths`.
- `uri` is an URI pattern that specifies the location of the distribution on the WWW. In this string any sub-string `"$(NAME)"` is replaced by the package name, `"$(VERSION)"` is replaced by the distribution version string and `"$(VERSION_NUM)"` by the distribution version string, chopping an initial `'v'` or `'V'` character if present. This argument is used to generate the `url` file of an opam package for the distribution; it will be deprecated in the future in favour of a `x-distrib-uri` field in the opam file. If the value is unspecified it defaults to:

      PKG_HOMEPAGE/releases/$(NAME)-$(VERSION_NUM).tbz

  where PKG_HOMEPAGE is the package's opam file `homepage` field. As a special case if the hostname of PKG_HOMEPAGE is `github` the following is used:

      PKG_DEV_REPO/releases/download/$(VERSION)/$(NAME)-$(VERSION_NUM).tbz

  where PKG_DEV_REPO is the package's opam file `dev-repo` field without the `.git` suffix and a possible `git+` prefix.

    val watermarks : watermark list

`watermarks` is the default list of watermarks. It has the following elements:

- `("NAME", `Name)`
- `("VERSION", `Version)`
- `("VERSION_NUM", `Version_num)`
- `("VCS_COMMIT_ID", `Vcs [`Commit_id])`
- `("PKG_MAINTAINER", `Opam (None, "maintainer", ", "))`
- `("PKG_AUTHORS", `Opam (None, "authors", ", ")`
- `("PKG_HOMEPAGE", `Opam (None, "homepage", " ")`
- `("PKG_ISSUES", `Opam (None, "bug-reports", " ")`
- `("PKG_DOC", `Opam (None, "doc", " "))`
- `("PKG_LICENSE", `Opam (None, "license", ", ")`
- `("PKG_REPO", `Opam (None, "dev-repo", " "))`

Prepending to the list overrides default definitions.

    val files_to_watermark : unit -> Topkg.fpath list Topkg.result

`files_to_watermark ()` is the default list of files to watermark. It is invoked in the distribution build directory and gets the set of tracked files of this directory from which it removes the files that end with `.eps`, `.flv`, `.gif`, `.ico`, `.jpeg`, `.jpg`, `.mov`, `.mp3`, `.mp4`, `.otf`, `.pdf`, `.png`, `.ps`, `.ttf`, `.woff`.

    val massage : unit -> unit Topkg.result

`massage` is the default distribution massaging function. It is invoked in the distribution build directory and does nothing.

    val exclude_paths : unit -> Topkg.fpath list Topkg.result

`exclude_paths ()` is the default list of paths to exclude from the distribution archive. It is invoked in the distribution build directory and returns the following static set of files.

    fun () -> Ok [".git"; ".gitignore"; ".gitattributes"; ".hg"; ".hgignore";
                  "build"; "Makefile"; "_build"]

## Distribution publication description

    type publish

The type for describing distribution publication.

    val publish : ?artefacts:[ `Alt of string | `Distrib | `Doc ] list ->
           unit -> publish

`publish ~artefacts ()` influences the distribution publication process performed by the `topkg` tool:

- `artefacts` defines which artefacts are published by an invocation of `topkg publish` without arguments (defaults to `[`Doc;`Distrib]`).

## Package description

    type std_file

The type for specifying a standard file.

    val std_file : ?install:bool -> Topkg.fpath -> std_file

`std_file ~install p` is a standard file `p` expressed relative to the distribution root directory. The file is automatically installed if `install` is `true` (default).

    type meta_file

The type for specifying an OCamlfind META file.

    val meta_file : ?lint:bool -> ?install:bool -> Topkg.fpath -> meta_file

`meta_file ~lint ~install p` is a META file `p` expressed relative to the distribution root directory. The file is automatically installed in the `Topkg.Pkg.lib` field if `install` is `true` (default). If `lint` is `true` (default), it is OCamlfind linted.

    type opam_file

The type for specifying an opam file.

    val opam_file : ?lint:bool ->
           ?lint_deps_excluding:string list option ->
           ?install:bool -> Topkg.fpath -> opam_file

`opam_file ~lint ~lint_deps_excluding ~install p` is an opam file `p` expressd relative to the distribution root directory such that:

- If `install` is `true` (default), it is automatically installed in the `Topkg.Pkg.lib` field.
- If `lint` is `true` (default), it is opam linted.
- If `lint_deps_excluding` is `Some excludes`, `topkg` checks that each of the opam package dependencies is mentioned as a root package in the OCamlbuild `_tags` file and vice-versa. The following package names are excluded from this test:
  - The packages names mentioned in `excludes`.
  - Package names that start with `"conf-"`
  - `Topkg_care.OCamlfind.base_packages`
  - `Topkg_care.Opam.ocaml_base_packages`
  If `None` the dependency check is disabled.

    val describe : ?delegate:Topkg.Cmd.t ->
           ?readmes:std_file list ->
           ?licenses:std_file list ->
           ?change_logs:std_file list ->
           ?metas:meta_file list ->
           ?opams:opam_file list ->
           ?lint_files:Topkg.fpath list option ->
           ?lint_custom:(unit -> Topkg.R.msg Topkg.result list) ->
           ?distrib:distrib ->
           ?publish:publish ->
           ?build:build ->
           string -> (Topkg.Conf.t -> install list Topkg.result) -> unit

`describe name install` describes a package named `name` with:

- `delegate`, the package delegate command to use. If unspecfied determined by the delegate lookup procedure, see `topkg help delegate` for more information.
- `readmes` are readme files, defaults to `[std_file "README.md"]`. Automatic install is in the `Topkg.Pkg.doc` field.
- `licenses` are license files, defaults to `[std_file "LICENSE.md"]`. Automatic install is in the `Topkg.Pkg.doc` field.
- `change_logs` are change logs, defaults to `[std_file "CHANGES.md"]`. The first file of the list is the one that is acted upon by the `topkg log` command. Automatic install is in the `Topkg.Pkg.doc` field.
- `metas` the package's ocamlfind META files, defaults to `[ meta_file "pkg/META" ]`.
- `opams` the package's opam package files, defaults to `[opam_file "opam"]`. The default opam file used by a package description depends on the package `name` (which can be overriden from the command line). The opam file lookup procedure selects the first path in `opams` whose filename is `(name ^ ".opam")` and, failing to do so, it fallbacks to an `"opam"` file at the root of the distribution.
- `lint_files` if `Some files`, ensures that all files mentioned in `readme`, `license`, `change_log`, `metas`, `opams` and `files` are present in the distribution. Defaults to `Some []`. If `None` disables the file existence tests (including readme, change_log, license, metas, opams, metas.)
- `lint_custom` defines a custom linting process run with the current directory set at the root of the distribution. Successes and errors in the returned list are reported as such and any error in the list makes the lint fail. Defaults to `None`.
- `distrib`, specifies the distribution process, defaults to `Topkg.Pkg.distrib``()`.
- `publish`, specifies the publication process, defaults to `Topkg.Pkg.publish``()`.
- `build`, specifies the build process, defaults to `Topkg.Pkg.build``()`.
- `install` given a build configuration specifies the install moves. Note that some of standard files are automatically installed and don't need to be specified, see `Topkg.Pkg.std_file`, `Topkg.Pkg.meta_file` and `Topkg.Pkg.opam_file`.

## Package distribution creation details

The following describes the exact steps performed by `topkg
distrib` to create the distribution archive. Note that `topkg` allows to override or disable part of the process via command line arguments, e.g. to specify the version string manually or skip linting. See `topkg distrib --help` for more information.

The distribution process assumes that the source repository working directory is clean so that its definitions are consistent with those of the distribution build directory. A warning is generated if this is not the case as it may end up in inconsistent distribution archives (but which may be fine to only publish a documentation update).

Let `$NAME` be the name of the package, `$BUILD` be its build directory, `$VERSION` be the VCS tag description (e.g. `git-describe(1)` if you are using `git`) of the source repository HEAD commit and `distrib` the distribution description found in the source's repository `pkg/pkg.ml` file.

1. Clone the source repository at `HEAD` as the distribution build directory `$BUILD/$NAME-$VERSION.build`.
2. Prepare the distribution:
   1. Invoke the `files_to_watermark` function of `distrib` in the distribution build directory to determine the files to watermark with `watermarks` and perform the watermarking process.
   2. Adds a version field with value `$VERSION` to the opam files mentioned by `Topkg.Pkg.describe`.
   3. Run the `massage` function of `distrib` in the distribution build directory. This can be used to create distribution time build artefacts.
3. Invoke the `exclude_paths` function of `distrib` in the distribution build directory to determine the paths to exclude from the archive.
4. Create a distribution tarball `$BUILD/$NAME-$VERSION.tbz` with the file hierarchy in `$BUILD/$NAME-$VERSION.build`, excluding the paths determined at the preceeding point and delete the clone `$BUILD/$NAME-$VERSION.build`. File modifications times in the archive are set to `HEAD`'s commit time and file permissions are preserved. Any other form of file metadata is discarded in the archive.
5. Test the distribution. Unpack it in directory `$BUILD/$NAME-$VERSION`, lint the distribution, build the package in the current build environment with its tests, run the tests, on success delete `$BUILD/$NAME-$VERSION`. Note that this uses the archive's `pkg/pkg.ml` file, which should not be different from the source's repository file if the latter was clean when `topkg distrib` was invoked.

### Note on watermarking

It is right to doubt the beauty and be concerned about the watermarking process. However experience shows that alternatives like having an OCaml module generated with the appropriate information doesn't work well in practice. Version numbers do not only show up in OCaml source code. They also appear in documentation comments, metadata files, textual data files and non-OCaml source files.

Watermarking by default all the non binary files of the distribution allows one to write %‌%VERSION%% in any context and be sure it is be substituted with the right version number in dev package (``Pin`) and distribution (``Distrib`) build contexts (this occurence was not subsituted because a ZERO WIDTH NON-JOINER U+200C was introduced between the first two percent characters).

If this scheme poses a problem for certain files or you remain unconvinced, simply filter the result of `Topkg.Pkg.files_to_watermark` or replace it by the exact files you would like to watermark.