# projtex: projection texturing API (agent guide)

> Give a 3D mesh and a prompt; get back the same model with a baked colour texture (GLB).
> Images are generated from virtual cameras by Qwen-Image-2.1 (official Qwen Space, default) or Bria Fibo 2.0
> Edit Turbo, projected onto the geometry, blended in texture space and baked into the mesh's UVs.
> Long operations are asynchronous jobs: create, then poll. Every status tells you what to do next.

Interfaces (same functionality):
- REST: base URL of this Space (https://townbox-projection-texturing.hf.space). OpenAPI/Swagger at /docs,
  JSON schema at /openapi.json.
- MCP (streamable HTTP, JSON responses): POST /mcp. Tools: texture_mesh, get_job, view_image, regenerate_views,
  disable_views, fill_gaps, retouch, undo_retouch, bake, pick_point, submit_view_image, service_info, guide.

## Quota and identity (read this first)
Generation runs on ZeroGPU Spaces and spends the caller's daily GPU quota (about 30 s of GPU per view).
- Send your Hugging Face token so your own quota is used: header `X-HF-Token: $HF_TOKEN` (REST and MCP), or the
  `hf_token` field. It is kept in memory for the job only, never stored, logged or returned.
- Without a token the job runs as your forwarded visitor identity if the Hub provides one, otherwise anonymously
  (shared ~2 GPU-minutes per day: about 4 views). GET /api/identity shows which applies to your requests.
- backend "mock" runs the whole pipeline with a procedural texture and no quota: use it to test your integration.
- backend "external" lets you supply the images yourself (any generator): no quota either.

## Minimal workflow (REST)
1. Send the mesh. A local file (e.g. exported from Blender or three.js): upload the raw bytes, then create the job
   with plain JSON (no multipart quoting problems, any characters in the prompt):
     curl -s --data-binary @house.glb -H 'content-type: model/gltf-binary' $BASE/api/uploads
     -> {"upload_id": "up_...", ...}
     curl -s -X POST $BASE/api/jobs -H 'content-type: application/json' -H "X-HF-Token: $HF_TOKEN" \
          -d '{"upload_id": "up_...", "prompt": "weathered red brick walls, dark slate roof; white window frames"}'
   Python: requests.post(f"{BASE}/api/uploads", data=open("house.glb", "rb")).json()["upload_id"], then
   requests.post(f"{BASE}/api/jobs", json={...}, headers={"X-HF-Token": token}).
   Other ways: "mesh_url" (public URL), "mesh_base64" (the file base64-encoded, up to 20 MB; handy over MCP),
   "sample" (avocado | lantern | head), or multipart POST /api/jobs/upload with field "file" plus either a "params"
   JSON field or separate fields (prompt, preset, ...). With curl -F, a value containing ';' is cut: use
   --form-string 'params=...' or -F 'params=<params.json'.
   -> 202 {"job_id": "...", "state": "queued", "next": [...], "poll_after_seconds": 15}
2. GET /api/jobs/{job_id} every `poll_after_seconds`. While running, `progress` shows the step, view i/N and
   `eta_seconds`; while queued, `queue` shows your position and an estimated start.
3. When state == "done":
   - files.textured_glb: your model with the baked texture (see "Using the result" below)
   - files.contact_sheet / files.comparison: renders to look at; files.albedo: the texture (PNG)
   - summary: coverage_pct, holes_pct, views_rejected, views_flagged_inconsistent (see "Reading results")
   - output: how the result maps onto your own scene (UV layout kept or not)
   - next: concrete follow-up calls (method, url, body) when quality can be improved
4. Optional quality loop (each is a job operation; poll again until done):
   - POST /api/jobs/{id}/regenerate {"views": [...]}   new seeds; failed regenerations keep the old image
   - POST /api/jobs/{id}/disable-views {"views": [...]} drop views that stay inconsistent
   - POST /api/jobs/{id}/fill-gaps {"max_views": 3}      cameras for still-uncovered surface
   - POST /api/jobs/{id}/retouch {"suggestion": 0}       local inpainting of the worst region
   - POST /api/jobs/{id}/bake {"resolution": 4096}

## Using the result (Blender, three.js, game engines)
- textured_glb has the same objects (node names and transforms) and materials (names, metallic, roughness, alpha)
  as the file you sent, in the same orientation; every material's base colour now uses one shared baked texture
  atlas. Normal/roughness/metallic maps are not produced. Import it in place of your model:
  Blender: bpy.ops.import_scene.gltf(filepath="textured.glb") (objects keep their names; the image is packed in
  the GLB). three.js: GLTFLoader; the texture is sRGB and already flagged as such; `getObjectByName` works with
  your original names.
- output.uv_layout tells whether your UVs were kept. If kept (output.albedo_matches_your_uvs), files.albedo also
  fits your original model directly: plug it into your own material's base colour. If they were regenerated
  (they overlapped or were missing, e.g. three.js BoxGeometry/CylinderGeometry, Blender primitives after joining),
  use the GLB, whose UVs are new. Force a choice with "uv": "keep" | "regen".
- Prepare meshes like this: one GLB with all parts (several objects and materials are fine), modifiers applied,
  faces pointing outwards, real thickness or separate parts rather than overlapping coplanar faces. UVs are
  optional (Blender: Smart UV Project is fine; or let the service unwrap).
- Orientation: glTF/GLB is always +Y up (Blender's glTF exporter converts its Z-up scene), so do not pass "up"
  for GLB files (mode "character" detects figures lying along Z by itself). Use "up": "z" only for OBJ/PLY/STL
  files whose up axis is Z.
Recipes:
  Blender (bpy, background mode):
    bpy.ops.export_scene.gltf(filepath="model.glb", export_format="GLB", export_apply=True)  # modifiers applied
    ... upload, create the job, poll, download files.textured_glb ...
    bpy.ops.import_scene.gltf(filepath="textured.glb")  # same object names; delete or hide the untextured ones
  three.js in Node (GLTFExporter needs FileReader, which Node lacks):
    globalThis.FileReader ??= class { readAsArrayBuffer(b) { b.arrayBuffer().then(r => { this.result = r;
      this.onloadend?.(); }); } readAsDataURL(b) { b.arrayBuffer().then(r => { this.result = `data:${b.type ||
      "application/octet-stream"};base64,${Buffer.from(r).toString("base64")}`; this.onloadend?.(); }); } };
    const glb = await new GLTFExporter().parseAsync(scene, { binary: true });
    fs.writeFileSync("model.glb", Buffer.from(glb));   // name your meshes and materials: the names come back
  three.js in the browser: new GLTFLoader().load("textured.glb", g => scene.add(g.scene)).

## Parameters that matter
- prompt: describe MATERIALS and COLOURS of what the mesh models ("olive drab chipped paint, gold visor"), not new
  geometry. Name the modelled parts (door, windows, roof, glasses...) so the model paints them.
- part_prompts (recommended when your file has several named materials or objects, e.g. a Blender scene):
  {"<material or object name>": "description"}, e.g. {"Walls": "white clapboard siding", "Roof": "grey asphalt
  shingles", "FrontDoor": "dark green panel door"}, with `prompt` kept for the overall style ("a tidy suburban
  house, late afternoon"). Each view's prompt then names only the parts that view shows, so a side view of a
  house is not given "a green door" and does not paint one on a blank wall. Names match material names first,
  then object names, case-insensitive; the status lists your file's names under model.materials /
  model.objects, and warnings lists names that were not found. Parts you leave out are still textured from
  `prompt`. Without part_prompts, meaningful material/object names are only used to keep out-of-view parts out
  of each picture (negative prompt).
- mode: object (orbit views; also buildings and props) | character (+ face close-ups, auto up-axis/facing; set
  focus_prompt to describe only the head) | environment (360° probes from inside, for rooms and interiors you
  walk through; slow on this CPU machine).
- preset: minimal (4 views, fastest) | balanced (7, default) | dense (12) | greedy. Gap views (up to 3) are added.
- backend: qwen-space (default) | fibo-space (most consistent views, best masked edits; CC BY-NC outputs) | mock |
  external.
- texture_size 1024 | 2048 | 4096; view_size 512-1024; uv auto | keep | regen; up (non-glTF files only).
- Name your parts in Blender/three.js (object.name, material.name): names come back in the GLB and can be used
  in part_prompts. Avoid default names (Cube.001, Material.002): they carry no meaning.
Time: ~(views x 30 s) + 1-2 min of CPU work here. Measured on this Space: 4 views ~3 min (qwen-space and
fibo-space); a balanced object job (7 views + up to 3 gap views) ~6-8 min. preset "minimal" is for quick tests:
it leaves much of a complex mesh uncovered; use balanced or dense for real work.

## Reading results
- summary.verdict: good (complete and consistent: use it) | usable (fine at a distance; `next` lists optional
  fixes) | needs work (follow `next`), with verdict_why.
- summary.coverage_pct / holes_pct are measured over the surface a camera can see from the side or above
  (summary.measured_over). The underside, visible only from below (e.g. the bottom of a crate standing on the
  ground), is reported apart as underside_pct / underside_covered_pct; uncovered texels are filled with nearby
  colours. Typical good results: coverage 85-95 %, holes < 10 %.
- last_result: outcome of the latest operation. retouch: status (applied | empty_mask | rejected), images.compare
  (before | model input | model output | after), validation attempts; regenerate/texture: per-view status;
  fill_gaps: coverage before/after and views added. If a retouch was not applied, `next` says how to aim it.
- views[].status: generated (used) | rejected (did not line up with the geometry after 3 attempts; excluded) |
  planned/pending. edge_agreement/silhouette_iou: alignment scores (good views ~0.3-0.7 / >0.85).
- summary.views_flagged_inconsistent: views whose content contradicts the others (e.g. a face painted on the back
  of a head). Regenerate them; disable if still flagged.
- retouch_suggestions: problem regions (uncovered / stretched / inconsistent) with an index for /retouch.
- warnings: parameter problems worth fixing (e.g. "up" given for a GLB).

## Retouch
POST /api/jobs/{id}/retouch with either {"suggestion": i} or an explicit camera + mask:
- camera: "view": "<view name>" | "render_camera": "orbit03" | "focus": {"point":[x,y,z],"radius":r}
- mask: "all" | "uncovered" | "low_quality" | {"sphere":[x,y,z,r]} | {"rect":[x0,y0,x1,y1]} | {"circle":[cx,cy,r]}
  (rect/circle in pixels or 0-1 fractions of the image)
- mode: repair | replace (+prompt) | upscale ({"capture_size":256,"output_size":1024}: re-imagine detail)
Use POST /api/jobs/{id}/pick {"render_camera":"orbit02","pixels":[[0.5,0.4]]} to turn a pixel you saw in a render
into a 3D point for focus/sphere masks. Retouches that repaint outside the mask or lose the geometry are rejected.

## Bring your own images (backend "external")
The job pauses with state "waiting_for_images" and `pending_view` = {view, condition_url, prompt, width, height}.
Generate an image with exactly the conditioning render's framing (paint the white clay, keep geometry), then
POST /api/jobs/{id}/views/{view}/image (multipart "file", or JSON {"image_url"} / {"image_base64"}). Views are
requested one at a time, in order, each conditioned on the previous ones.

## Errors
All errors are JSON {"error", "hint"}. 409 = the job is busy (poll first). 503 = queue full (retry later).
state "stopped" = ZeroGPU quota exhausted: retry later with `regenerate` on the remaining views, pass a token, or
switch backend to "external". Jobs, uploads and files expire after 12 h or when the Space restarts: download what
you need.

## Samples
avocado (object, CC0) | lantern (object with thin parts, CC0) | head (scanned bust, character, CC-BY 3.0
Infinite-Realities).