In this previous module, we did not load a single package: everything we wanted to do was provided by Julia “out of the box”. In most applications, we will need to get functionalities from other packages, and this is where Julia’s package manager shines.
There are two families of packages – some are part of the standard library, also known as Base, which is documented in the manual; others are developped by members of the community. Together, these packages form the broad “Julia ecosystem”.
The Julia package manager (also known as Pkg) is a complex but really well documented beast. For this reason, this module is primarily meant to point you towards the documentation, for everything related to installing and updating packages.
Nevertheless, there is one fundamental concept that is worth saying very explicitely: never work in the default environment. In most cases, this means that you will navigate to your working folder, and call Julia using
julia --project
Alternatively, after starting Julia, you can drop to the Pkg mode of the
REPL by pressing ], and calling the activate command. Most of the time,
this will be done as activate ., which means “activate an environment in the
current directory”.
. notation is a shortcut to mean here, .. means “the folder
immediately above here”, and ~ means “home”. In the vast majority of cases,
we will navigate our fileystem using Julia’s pathing utilities.Of course, because Pkg is a package, it can be called programmatically, i.e. as part of a script:
import Pkg
It offers the same commands as the package mode from the REPL; for example, we can activate an environemnt in a temporary directory with a nifty little trick:
Pkg.activate(mkpath(tempname()))
Activating new project at `/tmp/jl_G9wpMxu9K9`
This draws on concepts we will see (much) later, during the module on working with files and paths.
We can query the status of this environment:
Pkg.status()
Status `/tmp/jl_G9wpMxu9K9/Project.toml` (empty project)
This project is currently empty, meaning that we have not added dependencies (i.e. packages we will use). Let’s fix this, by installing the Distributions package:
Pkg.add("Distributions")
Updating registry at `~/.julia/registries/General.toml`
Resolving package versions...
Updating `/tmp/jl_G9wpMxu9K9/Project.toml`
[31c24e10] + Distributions v0.25.109
Updating `/tmp/jl_G9wpMxu9K9/Manifest.toml`
[66dad0bd] + AliasTables v1.1.3
[49dc2e85] + Calculus v0.5.1
[34da2185] + Compat v4.15.0
[9a962f9c] + DataAPI v1.16.0
[864edb3b] + DataStructures v0.18.20
[31c24e10] + Distributions v0.25.109
[ffbed154] + DocStringExtensions v0.9.3
[fa6b7ba4] + DualNumbers v0.6.8
[1a297f60] + FillArrays v1.11.0
[34004b35] + HypergeometricFunctions v0.3.23
[92d709cd] + IrrationalConstants v0.2.2
[692b3bcd] + JLLWrappers v1.5.0
[2ab3a3ac] + LogExpFunctions v0.3.28
[e1d29d7a] + Missings v1.2.0
[77ba4419] + NaNMath v1.0.2
[bac558e1] + OrderedCollections v1.6.3
[90014a1f] + PDMats v0.11.31
[21216c6a] + Preferences v1.4.3
[43287f4e] + PtrArrays v1.2.0
[1fd47b50] + QuadGK v2.9.4
[189a3867] + Reexport v1.2.2
[79098fc4] + Rmath v0.7.1
[a2af1166] + SortingAlgorithms v1.2.1
[276daf66] + SpecialFunctions v2.4.0
[82ae8749] + StatsAPI v1.7.0
[2913bbd2] + StatsBase v0.34.3
[4c63d2b9] + StatsFuns v1.3.1
[efe28fd5] + OpenSpecFun_jll v0.5.5+0
[f50d1b31] + Rmath_jll v0.4.2+0
[0dad84c5] + ArgTools v1.1.1
[56f22d72] + Artifacts
[2a0f44e3] + Base64
[ade2ca70] + Dates
[f43a241f] + Downloads v1.6.0
[7b1f6079] + FileWatching
[b77e0a4c] + InteractiveUtils
[b27032c2] + LibCURL v0.6.4
[76f85450] + LibGit2
[8f399da3] + Libdl
[37e2e46d] + LinearAlgebra
[56ddb016] + Logging
[d6f4376e] + Markdown
[ca575930] + NetworkOptions v1.2.0
[44cfe95a] + Pkg v1.10.0
[de0858da] + Printf
[3fa0cd96] + REPL
[9a3f8284] + Random
[ea8e919c] + SHA v0.7.0
[9e88b42a] + Serialization
[6462fe0b] + Sockets
[2f01184e] + SparseArrays v1.10.0
[10745b16] + Statistics v1.10.0
[4607b0f0] + SuiteSparse
[fa267f1f] + TOML v1.0.3
[a4e569a6] + Tar v1.10.0
[cf7118a7] + UUIDs
[4ec0a83e] + Unicode
[e66e0078] + CompilerSupportLibraries_jll v1.1.1+0
[deac9b47] + LibCURL_jll v8.4.0+0
[e37daf67] + LibGit2_jll v1.6.4+0
[29816b5a] + LibSSH2_jll v1.11.0+1
[c8ffd9c3] + MbedTLS_jll v2.28.2+1
[14a3606d] + MozillaCACerts_jll v2023.1.10
[4536629a] + OpenBLAS_jll v0.3.23+4
[05823500] + OpenLibm_jll v0.8.1+2
[bea87d4a] + SuiteSparse_jll v7.2.1+1
[83775a58] + Zlib_jll v1.2.13+1
[8e850b90] + libblastrampoline_jll v5.8.0+1
[8e850ede] + nghttp2_jll v1.52.0+1
[3f19e933] + p7zip_jll v17.4.0+2
Precompiling project...
✓ DataAPI
✓ PtrArrays
✓ NaNMath
✓ StatsAPI
✓ CompilerSupportLibraries_jll
✓ FillArrays
✓ Reexport
✓ IrrationalConstants
✓ Calculus
✓ DocStringExtensions
✓ SortingAlgorithms
✓ QuadGK
✓ Statistics
✓ SuiteSparse
✓ Rmath_jll
✓ Missings
✓ AliasTables
✓ OpenSpecFun_jll
✓ LogExpFunctions
✓ PDMats
✓ Rmath
✓ FillArrays → FillArraysPDMatsExt
✓ FillArrays → FillArraysSparseArraysExt
✓ StatsBase
✓ FillArrays → FillArraysStatisticsExt
✓ SpecialFunctions
✓ DualNumbers
✓ HypergeometricFunctions
✓ StatsFuns
✓ Distributions
30 dependencies successfully precompiled in 14 seconds. 10 already precompiled.
Note that the package manager informs us on the specific dependencies of the package we want to install. This management of the network of dependencies is largely invisible to the user, but is reported at install (and removal) time.
We can check that it has been installed:
Pkg.status()
Status `/tmp/jl_G9wpMxu9K9/Project.toml`
[31c24e10] Distributions v0.25.109
import Pkg without installing Pkg first
is that Pkg is a part of the standard library, and comes bundled with
Julia. It is, in fact, one of the most central pieces of the language.We can now use the Distributions package:
using Distributions
Normal(0.0, 0.2)
Distributions.Normal{Float64}(μ=0.0, σ=0.2)
In some cases, we may want to remove an existing package from our environment, because we have found an alternative solution. It is good practice to only keep the dependencies you actually need and use.
Pkg.rm("Distributions");
Pkg.status("Distributions")
Updating `/tmp/jl_G9wpMxu9K9/Project.toml`
[31c24e10] - Distributions v0.25.109
Updating `/tmp/jl_G9wpMxu9K9/Manifest.toml`
[66dad0bd] - AliasTables v1.1.3
[49dc2e85] - Calculus v0.5.1
[34da2185] - Compat v4.15.0
[9a962f9c] - DataAPI v1.16.0
[864edb3b] - DataStructures v0.18.20
[31c24e10] - Distributions v0.25.109
[ffbed154] - DocStringExtensions v0.9.3
[fa6b7ba4] - DualNumbers v0.6.8
[1a297f60] - FillArrays v1.11.0
[34004b35] - HypergeometricFunctions v0.3.23
[92d709cd] - IrrationalConstants v0.2.2
[692b3bcd] - JLLWrappers v1.5.0
[2ab3a3ac] - LogExpFunctions v0.3.28
[e1d29d7a] - Missings v1.2.0
[77ba4419] - NaNMath v1.0.2
[bac558e1] - OrderedCollections v1.6.3
[90014a1f] - PDMats v0.11.31
[21216c6a] - Preferences v1.4.3
[43287f4e] - PtrArrays v1.2.0
[1fd47b50] - QuadGK v2.9.4
[189a3867] - Reexport v1.2.2
[79098fc4] - Rmath v0.7.1
[a2af1166] - SortingAlgorithms v1.2.1
[276daf66] - SpecialFunctions v2.4.0
[82ae8749] - StatsAPI v1.7.0
[2913bbd2] - StatsBase v0.34.3
[4c63d2b9] - StatsFuns v1.3.1
[efe28fd5] - OpenSpecFun_jll v0.5.5+0
[f50d1b31] - Rmath_jll v0.4.2+0
[0dad84c5] - ArgTools v1.1.1
[56f22d72] - Artifacts
[2a0f44e3] - Base64
[ade2ca70] - Dates
[f43a241f] - Downloads v1.6.0
[7b1f6079] - FileWatching
[b77e0a4c] - InteractiveUtils
[b27032c2] - LibCURL v0.6.4
[76f85450] - LibGit2
[8f399da3] - Libdl
[37e2e46d] - LinearAlgebra
[56ddb016] - Logging
[d6f4376e] - Markdown
[ca575930] - NetworkOptions v1.2.0
[44cfe95a] - Pkg v1.10.0
[de0858da] - Printf
[3fa0cd96] - REPL
[9a3f8284] - Random
[ea8e919c] - SHA v0.7.0
[9e88b42a] - Serialization
[6462fe0b] - Sockets
[2f01184e] - SparseArrays v1.10.0
[10745b16] - Statistics v1.10.0
[4607b0f0] - SuiteSparse
[fa267f1f] - TOML v1.0.3
[a4e569a6] - Tar v1.10.0
[cf7118a7] - UUIDs
[4ec0a83e] - Unicode
[e66e0078] - CompilerSupportLibraries_jll v1.1.1+0
[deac9b47] - LibCURL_jll v8.4.0+0
[e37daf67] - LibGit2_jll v1.6.4+0
[29816b5a] - LibSSH2_jll v1.11.0+1
[c8ffd9c3] - MbedTLS_jll v2.28.2+1
[14a3606d] - MozillaCACerts_jll v2023.1.10
[4536629a] - OpenBLAS_jll v0.3.23+4
[05823500] - OpenLibm_jll v0.8.1+2
[bea87d4a] - SuiteSparse_jll v7.2.1+1
[83775a58] - Zlib_jll v1.2.13+1
[8e850b90] - libblastrampoline_jll v5.8.0+1
[8e850ede] - nghttp2_jll v1.52.0+1
[3f19e933] - p7zip_jll v17.4.0+2
Info We haven't cleaned this depot up for a bit, running Pkg.gc()...
Active manifest files: 2 found
Active artifact files: 61 found
Active scratchspaces: 0 found
Deleted no artifacts, repos, packages or scratchspaces
Status `/tmp/jl_G9wpMxu9K9/Project.toml` (empty project)
; at the end of the line. This works
for all commands, and is a great way to prevent some output from taking up too
much space.Another thing that the package manager can do is garbage collection, i.e. cleaning the versions and environments that are not used anymore:
Pkg.gc()
Active manifest files: 2 found
Active artifact files: 61 found
Active scratchspaces: 0 found
Deleted no artifacts, repos, packages or scratchspaces
This will look through all the environments Julia knows about, and reclaim disk space by removing versions of packages that are not used anymore, or dependencies of projects that have not been used for a long time.
There is a lot more that you can achieve using the package manager, including creating your own packages. Reading the documentation is key here, and will make maintaining your projects far more efficient.