An overview of the NuGet v3 feed design along with examples of the services used by the NuGet client.
Features
Versioned services
NuGet v3 feeds provide a collection of versioned services for the client. Clients supporting the latest services can benefit from the extra metadata or optimized format while older clients can continue using the endpoint they were designed against without the risk of breaking changes.
Stability
One of the most critical scenarios for NuGet is package restore, builds are blocked when a package fails to download. To make this scenario as robust and scalable as possible the v3 format is optimized for reading, looking up a package and downloading it can be done by utilizing only static content from the server. Removing additional server compute and database queries eliminates the most common failure points hit with v2 feeds.
Transaction log
A core component of the v3 feed is the package catalog. A record of all package adds and edits are recorded in an append only format. Discovering all packages on a feed and the history of each can be done by reading through the static json files that make up the catalog.
Json-LD
Feed content is stored in Json-LD, making it readable either as standard json or as RDF.
Components
These are the components that make up a v3 feed.
Service index
The main entry point for a v3 feed is the service index which contains a high level list of the services provided by the feed. All relevant URIs for getting to the rest of the services. For nuget.org this file is found at https://api.nuget.org/v3/index.json
Catalog
The catalog is an append only transaction log of all package adds and edits for the feed. It contains almost all of the metadata for each package. Other pages such as the package registrations are subset views of the package details pages contained in the catalog.
The catalog is primarily used by the feed itself to index packages, it is not used anywhere in the client today.
Service index entry
The feed service index contains an entry for the catalog index.
{
"@id": "https://api.nuget.org/v3/catalog0/index.json",
"@type": "Catalog/3.0.0",
"comment": "Index of the NuGet package catalog."
}
Catalog hierarchy
- Catalog index
- Catalog page
- Package details
Catalog index
The catalog index page contains a list of catalog pages. Below is an abbreviated example showing a single page in the list that we will follow.
{
"@id": "https://api.nuget.org/v3/catalog0/index.json",
"items": [
{
"@id": "https://api.nuget.org/v3/catalog0/page1586.json",
"@type": "CatalogPage",
"commitId": "dbb65239-0b86-4d38-88a2-f09fa0137f98",
"commitTimeStamp": "2016-04-26T18:29:57.9381745Z",
"count": 549
}
]
}
Catalog pages
Within the catalog page there is another list, this time of PackageDetails pages. Each entry here represents a package that was pushed or modified on the feed. The entry contains the id, version, time of the operation, and a link to the details page.
{
"@id": "https://api.nuget.org/v3/catalog0/page1586.json",
"@type": "CatalogPage",
"commitId": "dbb65239-0b86-4d38-88a2-f09fa0137f98",
"commitTimeStamp": "2016-04-26T18:29:57.9381745Z",
"count": 549,
"items": [
{
"@id": "https://api.nuget.org/v3/catalog0/data/2016.04.26.06.39.18/sleet.0.1.0-alpha-01.json",
"@type": "nuget:PackageDetails",
"commitId": "1e04024b-d11e-4f5c-83fd-a763a9971ca5",
"commitTimeStamp": "2016-04-26T06:39:18.6579353Z",
"nuget:id": "Sleet",
"nuget:version": "0.1.0-alpha-01"
}
]
}
Package details page
The lowest level is the package details page. This contains all metadata from the package nuspec along with a list of files found in the nupkg. Below is an abbreviated example.
{
"@id": "https://api.nuget.org/v3/catalog0/data/2016.04.26.06.39.18/sleet.0.1.0-alpha-01.json",
"@type": [
"PackageDetails",
"catalog:Permalink"
],
"catalog:commitId": "1e04024b-d11e-4f5c-83fd-a763a9971ca5",
"catalog:commitTimeStamp": "2016-04-26T06:39:18.6579353Z",
"created": "2016-04-26T06:38:56.613Z",
"id": "Sleet",
"isPrerelease": true,
"lastEdited": "0001-01-01T00:00:00Z",
"listed": true,
"packageHash": "ugr5yDLhm+gu5ryIkQ7gdCW+2t0vfmMqx4cqxdjabLQ+8vHt9x7w5SkW60mIJ5F8DDncXukN9sGoFtgY8Mcf8Q==",
"packageHashAlgorithm": "SHA512",
"packageSize": 1570425,
"projectUrl": "https://github.com/emgarten/Sleet",
"published": "2016-04-26T06:38:56.613Z",
"requireLicenseAcceptance": false,
"version": "0.1.0-alpha-01",
"packageEntries": [
{
"@id": "https://api.nuget.org/v3/catalog0/data/2016.04.26.06.39.18/sleet.0.1.0-alpha-01.json#Sleet.nuspec",
"@type": "PackageEntry",
"compressedLength": 351,
"fullName": "Sleet.nuspec",
"length": 618,
"name": "Sleet.nuspec"
}
],
"tags": [
"nuget",
"nugetserver",
"nugetfeed"
]
}
Package edits
Some nuspec fields can be modified through the nuget.org web site such as the project url. When a package is changed a new entry and package details page is added to the catalog.
Traversing the catalog
When searching for details on a package it is best to take the most recent entry since it will be the one that matches the current nupkg available for download. Many packages contain multiple entries since it is a common practice to upload the package first and then modify it through the web site.
The catalog does not have a way to construct urls directly to a specific package, the only way to search it is by reading through the catalog pages.
Registrations
Service index entry
{
"@id": "https://api.nuget.org/v3/registration1-gz/",
"@type": "RegistrationsBaseUrl/3.4.0"
},
{
"@id": "https://api.nuget.org/v3/registration1/",
"@type": "RegistrationsBaseUrl"
}
This service has multiple versions on nuget.org. NuGet client 3.4.0 and up use RegistrationsBaseUrl/3.4.0 if it exists, otherwise the RegistrationsBaseUrl type is used. The 3.4.0 version introduced gzip compression, dramatically reducing the download size for the client.
Registration index
The package registration index is a view of the latest package details for every version of a package id. This page is used by the NuGet Visual Studio client to populate the details pane in the UI, and also by the packages.config resolver to find dependency information for a package.
Below is a trimmed down registration page for a package with no dependencies.
{
"@id": "https://api.nuget.org/v3/registration1/sleet/index.json",
"count": 1,
"items": [
{
"@id": "https://api.nuget.org/v3/registration1/sleet/index.json#page/0.1.0-alpha-01/0.1.0-alpha-01",
"@type": "catalog:CatalogPage",
"count": 1,
"items": [
{
"@id": "https://api.nuget.org/v3/registration1/sleet/0.1.0-alpha-01.json",
"@type": "Package",
"catalogEntry": {
"@id": "https://api.nuget.org/v3/catalog0/data/2016.04.26.06.39.18/sleet.0.1.0-alpha-01.json",
"@type": "PackageDetails",
"description": "Sleet is a cross platform command line tool to generate NuGet v3 static feeds.",
"id": "Sleet",
"listed": true,
"packageContent": "https://api.nuget.org/packages/sleet.0.1.0-alpha-01.nupkg",
"published": "2016-04-26T06:38:56.613+00:00",
"requireLicenseAcceptance": false,
"version": "0.1.0-alpha-01"
},
"packageContent": "https://api.nuget.org/packages/sleet.0.1.0-alpha-01.nupkg",
"registration": "https://api.nuget.org/v3/registration1/sleet/index.json"
}
],
"parent": "https://api.nuget.org/v3/registration1/sleet/index.json",
"lower": "0.1.0-alpha-01",
"upper": "0.1.0-alpha-01"
}
]
}
As you can see this information is similar to the catalog's package details page, however this view allows the client to fetch multiple versions from a single URL.
This page has a similar structure to the catalog pages, there is an index, a second level of pages, and then entries within the pages. The difference is that here everything is on a single page for most packages.
For packages with hundreds of versions the pages are moved into external pages to keep the file sizes down and to allow for a package to have potentially 100,000 versions or more.
Package pages
The registration index for a package links to the package page, and vice versa. The package page is designed to be a permalink for a package id/version and contains links to both the registration page and the catalog details page.
{
"@id": "https://api.nuget.org/v3/registration1/sleet/0.1.0-alpha-01.json",
"@type": [
"Package",
"http://schema.nuget.org/catalog#Permalink"
],
"catalogEntry": "https://api.nuget.org/v3/catalog0/data/2016.04.26.06.39.18/sleet.0.1.0-alpha-01.json",
"listed": true,
"packageContent": "https://api.nuget.org/packages/sleet.0.1.0-alpha-01.nupkg",
"published": "2016-04-26T06:38:56.613+00:00",
"registration": "https://api.nuget.org/v3/registration1/sleet/index.json"
}
Registration URL templates
Registration urls may be constructed with the lowercase package id and normalized version.
These examples use the current base url https://api.nuget.org/v3/registration1/ however this url should always come from the service index since it may change in the future.
Registration index
https://api.nuget.org/v3/registration1/{id-lower}/index.json
Package page
https://api.nuget.org/v3/registration1/{id-lower}/{version-lower}.json
PackageBaseAddress
PackageBaseAddress is designed to be a very simple format with easily constructable urls similar the project.json packages folder structure.
This service is used by project.json restore for UWP and .NET Core projects.
Service index entry
{
"@id": "https://api.nuget.org/v3-flatcontainer/",
"@type": "PackageBaseAddress/3.0.0"
}
Version index
The list of available package versions can be discovered through the index file. This file contains a single array of the version strings. Note that this contains both listed and unlisted packages.
{
"versions": [
"1.0.1-beta",
"2.0.0"
]
}
URL templates
In the templates below refers to the lowercase package id. refers to the lowercase and normalized package version.
Version index
https://api.nuget.org/v3-flatcontainer/{id-lower}/index.json
nupkg download url
https://api.nuget.org/v3-flatcontainer/{id-lower}/{version-lower}/{id-lower}.{version-lower}.nupkg
nuspec url
https://api.nuget.org/v3-flatcontainer/{id-lower}/{version-lower}/{id-lower}.nuspec
Search
Search is used by the NuGet Visual Studio UI.
Service index entry
{
"@id": "https://api-v2v3search-0.nuget.org/query",
"@type": "SearchQueryService"
}
Search URL template
https://api-v2v3search-0.nuget.org/query?q={string}&skip={integer}&take={integer}&prerelease={boolean}&includeDelisted={boolean}&packageTypeFilter={string}
packageTypeFilter may appear multiple times.
The resulting json search results contains metadata similar to the registration pages. All listed package versions will be contained in the result along with a link to the registration index.
Search example
https://api-v2v3search-0.nuget.org/query?q=sleet&prerelease=true
{
"totalHits": 1,
"lastReopen": "2016-06-27T02:19:30.2881113Z",
"index": "v3-lucene0-v2v3-20160520",
"data": [
{
"@id": "https://api.nuget.org/v3/registration0/sleet/index.json",
"@type": "Package",
"registration": "https://api.nuget.org/v3/registration0/sleet/index.json",
"id": "Sleet",
"version": "0.1.0-alpha-01",
"description": "Sleet is a cross platform command line tool to generate NuGet v3 static feeds.",
"summary": "",
"title": "Sleet",
"projectUrl": "https://github.com/emgarten/Sleet",
"tags": [
"nuget",
"nugetserver",
"nugetfeed"
],
"authors": [
"juste"
],
"totalDownloads": 20,
"versions": [
{
"version": "0.1.0-alpha-01",
"downloads": 20,
"@id": "https://api.nuget.org/v3/registration0/sleet/0.1.0-alpha-01.json"
}
]
}
]
}
Supported clients
NuGet 3.0.0 and dotnet restore for .NET Core 1.0.0 preview tools and up support the NuGet v3 feed format.
Json-LD
Many parts of the v3 feed use json-ld. Json-ld adds extra properties on top of the json to allow parsing the json into RDF triples. One of the nice features of json-ld is that RDF isn't a requirement, the json can still be read normally and the properties with an @ prefix may be ignored. For the purposes of this post I've dropped many of them already including the @context sections, however when using an actual v3 feed you may notice these properties on every page.
The json-ld library used by NuGet is json-ld.net.
Normalized Versions
Feed urls use lowercase package ids, and versions that are both lowercase and normalized. A normalized version follows SemVer.org rules with a few exceptions for legacy NuGet versions. See NuGet.Versioning for help normalizing and constructing these strings.
Example of normalized versions
Original version: 1.0.01 Normalized: 1.0.1
Original version: 1.0 Normalized: 1.0.0
Original version: 1.0.0.0 Normalized: 1.0.0
Original version: 1.0.0.1 Normalized: 1.0.0.1"}]],"sections":[[10,0]]}",