Docs/Reference/Modules/rules_huggingface

rules_huggingface

Bazel-idiomatic HuggingFace toolchain: hf_model / hf_upload / hf_inference_endpoint

Latest	`0.0.4`
Versions	2
Category	Bazel rules
Maintainers	Matt Marshall
Registry	https://registry.tbzl.dev/modules/rules_huggingface/
Source	github.com/tomato-bazel/rules_huggingface

MODULE.bazelstarlark

bazel_dep(name = "rules_huggingface", version = "0.0.4")

View source & releases on GitHub ↗

Bazel-native HuggingFace Hub model + dataset push/pull (hf_model, hf_upload)

Status: v0.0.1 — scaffold

No public surface yet. See CHANGELOG.md for what has shipped.

Install

.bazelrc:

common --registry=https://registry.fastverk.com/
common --registry=https://bcr.bazel.build/

MODULE.bazel:

bazel_dep(name = "rules_huggingface", version = "0.0.1")

Usage#

Real usage, taken from the module’s examples/.

examples/BUILD.bazel

load(
    "@rules_huggingface//huggingface:defs.bzl",
    "hf_download",
    "hf_inference_endpoint",
    "hf_model",
    "hf_repo",
    "hf_upload",
)

# Smoke targets — verify the rules load + analyze + emit runners.

hf_model(
    name = "qwen_base",
    repo = "Qwen/Qwen2.5-1.5B-Instruct",
    revision = "main",
)

# → emits :demo_model.push
hf_upload(
    name = "demo_model",
    repo = "fastverk/demo-model",
    local_dir = "outputs/demo",
    private = True,
)

# → emits :demo_repo.create
hf_repo(
    name = "demo_repo",
    repo = "fastverk/demo-model",
    private = True,
)

# → emits :demo_fetch.download
hf_download(
    name = "demo_fetch",
    repo = "Qwen/Qwen2.5-1.5B-Instruct",
    local_dir = "outputs/qwen",
    files = ["config.json"],
)

# → emits :demo_endpoint.deploy/.pause/.resume/.scale_to_zero/.delete/.describe/.list
hf_inference_endpoint(
    name = "demo_endpoint",
    namespace = "fastverk",
    endpoint_name = "agora-parser",
    config = "endpoint_config.json",
)

Conformance#

3 findings across 2 invariants. 10 contested atoms. See how gating works or the full report.

D2 a non-dev register_toolchains propagates to every transitive consumer why this matters ↗

version	toolchain
`0.0.4`	`//huggingface:hf_default_toolchain`
`0.0.4`	`@rust_toolchains//:all`

D3 a repo name CHOSEN on a SHARED extension must be namespaced why this matters ↗

repo	extension
`hf_crates`	`@rules_rust//crate_universe:extension.bzl`

Contested atoms

Third-party modules where this module resolves a different version than others do. Not a violation of anything this module did — it is the actionable form of a registry-level convergence finding, and the sentence a maintainer can act on.

Atom	Resolved here	Elsewhere
`apple_support`	`1.24.2`	`2.2.0` ×1
`bazel_skylib`	`1.8.2`	`1.9.0` ×2
`gazelle`	`0.36.0`	`0.30.0` ×5`0.44.0` ×1`0.51.0` ×3
`nlohmann_json`	`3.6.1`	`3.12.0.bcr.1` ×1
`protobuf`	`33.4`	`34.0.bcr.1` ×2
`rules_go`	`0.60.0`	`0.39.1` ×5
`rules_jvm_external`	`6.7`	`6.8` ×4
`rules_python`	`1.7.0`	`2.0.1` ×1
`rules_swift`	`3.1.2`	`3.6.1` ×1
`upb`	`0.0.0-20220923-a547704`	`0.0.0-20230516-61a97ef` ×1

Dependencies#

rules_huggingface in the registry graph — what it depends on (left) and what depends on it (right).

Depends on

platforms1.0.0bazel_skylib1.8.2rules_shell0.6.1rules_python1.7.0rules_openapi0.2.1rules_rust0.70.0stardoc0.7.2dev

Versions#

2 published versions, newest first. Each resolves to an immutable, integrity-checked archive.

Version	Integrity (sha256)	Source archive
`0.0.4` latest	`7IRxl7VxSQIQicmc…`	tag archive ↗
`0.0.3`	`Es7wr682ywLGZH/Y…`	tag archive ↗

Changelog#

All notable changes to rules_huggingface. The format is loosely Keep a Changelog — version headers mirror the published bazel-registry entries.

0.0.3 — build-time hermetic dataset fetch

hf_dataset_repository repository rule + a huggingface bzlmod module extension (huggingface.dataset tag class). Sha-pinned fetch of a single file from a HF dataset/model repo via the resolve/<revision>/<file> URL, with optional archive extraction and a BUILD overlay — the build-time, content-addressed counterpart to the run-time hf_download. Intended for hermetic build inputs (e.g. benchmark archives like ToolBench’s ToolEnv tarball pinned into a consumer’s MODULE.bazel). This is the “repository-rule variant” flagged as future work in 0.0.2’s hf_download docs.

0.0.2 — hermetic `hf` toolchain + data-plane rules

Hermetic hf CLI. huggingface_hub==1.16.1 served from a pinned, universal, hashed pip hub (@hf_pypi) via py_console_script_binary — no system hf/uvx. (The [cli] extra is gone in huggingface_hub 1.x; the CLI ships in the base package.)
hf toolchain. //huggingface:toolchain_type + a default hf_default_toolchain. Consumers can register_toolchains to pin a different huggingface_hub version without forking the rules.
Data-plane rules (resolve the toolchain, emit a bazel run-able runner per the cluster verb-suffix convention):
- hf_upload → <name>.push (create-if-missing + sync a dir)
- hf_repo → <name>.create (create / no-op reuse a repo)
- hf_download → <name>.download (materialize a repo/files locally)
hf_model retained as a typed HfModelInfo repo reference.
Rationale: the data-plane protocol (Git LFS + Xet + the commit endpoint) is bespoke client logic only huggingface_hub implements, so the binary is vendored rather than codegen’d.
Control plane — hf_inference_endpoint. The Inference Endpoints REST API is a clean OpenAPI 3.1 surface, so (unlike the data plane) it’s a typed Rust client codegen’d by rules_openapi (progenitor) from the vendored spec, wrapped by the hf-endpoints CLI. Emits <name>.deploy (config-file body deserialized + validated against the generated types::Endpoint) plus .pause / .resume / .scale_to_zero / .delete / .describe / .list. A hermetic 3.1→3.0 down-converter (openapi/downconvert.py) bridges progenitor’s 3.0-only parser. One isolated @hf_crates universe backs both the client and the CLI. (Required upstreaming rules_openapi 0.2.1: threadable chrono/uuid/bytes.)

0.0.1 — hf_model + hf_upload

First public API: hf_model (typed ref) + hf_upload (macro emitting an sh_binary .push runner over a system hf).

← All modules