> ## Documentation Index
> Fetch the complete documentation index at: https://docs.apimart.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Grok Imagine 2.0 Ext Layers and Region Editing

> Use segment to retrieve object layers and precise masks, then edit polygons, boxes, or detected objects with region_edit.

<Info>
  Both `segment` and `region_edit` use the existing asynchronous image endpoint. Save the returned `task_id`, then poll [Get task status](/en/api-reference/tasks/status); the create request does not return final layers or images.
</Info>

<Warning>
  Never expose an API key in a browser bundle, LocalStorage, a URL, or frontend logs. Call APIMart through your backend or BFF.
</Warning>

<RequestExample>
  ```bash cURL theme={null}
  curl --request POST \
    --url https://api.apimart.ai/v1/images/generations \
    --header 'Authorization: Bearer <token>' \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    --header 'Idempotency-Key: 6baf0940-25d6-4ec2-9131-925250840fa7' \
    --data '{
      "model": "grok-imagine-2.0-ext",
      "operation": "segment",
      "image_urls": ["https://upload.apimart.ai/f/image/..."],
      "cache_only": true,
      "include_mask_rle": false
    }'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 theme={null}
  {
    "code": 200,
    "data": [{ "status": "submitted", "task_id": "task_..." }]
  }
  ```
</ResponseExample>

## Operation overview

| Purpose | Key input | Completed result | Billing |
| - | - | - | - |
| `segment`: Detect objects and retrieve layers, boxes, and precise masks | `source_task_id`, or `image_urls` containing one uploaded image | `image_id`, `image_url`, `objects` | Free |
| `region_edit`: Edit a polygon, rectangle, or detected object | `image_id`, `prompt`, selection | New URL and `image_id` | Charged per completed task |

```text theme={null}
completed task_id ───────┐
                         ├→ segment → image_id + mask_rle
uploaded public image URL┘           → selection_regions → region_edit → new task_id + image_id
```

<Note>
  Choose exactly one `segment` source: `source_task_id` or `image_urls`. These fields and `image_id` are not interchangeable; `region_edit` still takes the asset ID returned by `segment`. To segment an edited image, use the completed `region_edit` task ID as the next `source_task_id`.
</Note>

## Request headers

Use `Authorization: Bearer <APIMART_API_KEY>`, `Content-Type: application/json`, and `Accept: application/json`.

`Idempotency-Key` is optional and strongly recommended for paid `region_edit` requests. It accepts 1–191 visible ASCII characters; UUID is recommended. Use a new key for each new logical operation. A network retry of the same request must reuse the original key and identical body. If a paid edit has an indeterminate result, do not retry automatically with a new key.

## Asynchronous task flow

A successful create request returns HTTP `200` and `data[0].task_id`. Poll `GET /v1/tasks/{task_id}?language=en` every 2 seconds, back off to at most 5 seconds, and set a 10-minute overall timeout. Stop old polling when the source image changes.

<Warning>
  A task query can return HTTP `200` while `data.status` is `failed`. Always determine success from `data.status` and display `data.error` when present.
</Warning>

## `segment`

### Request parameters

| Field | Type | Required | Description |
| - | - | :-: | - |
| `model` | string | ✅ | Fixed to `grok-imagine-2.0-ext` |
| `operation` | string | ✅ | Fixed to `segment` |
| `nsfw_check` | boolean | — | Default: `false`.<br />`true`: use `omni-moderation-latest` to review the source image.<br />`false` or omitted: do not send a moderation request. |
| `source_task_id` | string | Conditional | A completed, single-image Grok task owned by the current user; mutually exclusive with `image_urls` |
| `image_urls` | string\[] | Conditional | Exactly one publicly accessible absolute HTTP(S) URL; mutually exclusive with `source_task_id`. Upload local images with `POST /v1/uploads/images`, then use the returned `url` |
| `include_mask_rle` | boolean | — | Default: `true`; `false` omits RLE masks but still returns the asset ID, object indexes, and boxes |
| `cache_only` | boolean | — | Default: `false`; required as `true` with `image_urls`; check segmentation cache only and do not call upstream on a miss |
| `cached_only` | boolean | — | Default: `false`; source-task-only upstream cache hint, not a local cache guarantee |
| `refresh` | boolean | — | Default: `false`; source-task-only cache bypass; do not use in the normal editor flow |

`segment` does not need `prompt`. Do not send `image_id`, `image_index`, `billing_model_name`, `n`, `size`, or `response_format`. Send exactly one of `source_task_id` and `image_urls`. Image URL mode requires `cache_only=true` and does not support `cached_only` or `refresh`.

### Request examples

<Tabs>
  <Tab title="Use task ID">
    ```json theme={null}
    {
      "model": "grok-imagine-2.0-ext",
      "operation": "segment",
      "source_task_id": "<COMPLETED_SINGLE_IMAGE_TASK_ID>",
      "include_mask_rle": true,
      "cached_only": false
    }
    ```
  </Tab>

  <Tab title="Use uploaded image">
    ```json theme={null}
    {
      "model": "grok-imagine-2.0-ext",
      "operation": "segment",
      "image_urls": ["https://upload.apimart.ai/f/image/..."],
      "cache_only": true,
      "include_mask_rle": false
    }
    ```
  </Tab>

  <Tab title="Task cache probe">
    ```json theme={null}
    {
      "model": "grok-imagine-2.0-ext",
      "operation": "segment",
      "source_task_id": "<COMPLETED_SINGLE_IMAGE_TASK_ID>",
      "include_mask_rle": true,
      "cache_only": true
    }
    ```
  </Tab>
</Tabs>

A cache miss is still a successful task. Use `cache_status` (`hit` or `miss`) or `from_cache`; do not infer a hit from `cached`.

### Upload a local image

Upload the local file first and read the public URL from the response:

```bash theme={null}
curl --request POST \
  --url https://api.apimart.ai/v1/uploads/images \
  --header 'Authorization: Bearer <token>' \
  --form 'file=@/path/to/source.png'
```

Pass the returned `url` as the only item in `image_urls`. Polling, the completed response, and `region_edit` then work exactly as with a task ID: read `result.image_id` and `objects`, then submit the selection edit. Uploaded URLs are temporary and retained for 72 hours by default.

<Warning>
  `image_urls` accepts exactly one publicly accessible absolute HTTP(S) URL. Image URL mode supports only `cache_only=true`; do not also send `source_task_id`, `cached_only`, or `refresh`.
</Warning>

### Completed response

For `segment`, `data.result` is the segmentation result directly; it is not wrapped in `images`.

```json theme={null}
{
  "code": 200,
  "data": {
    "id": "task_...",
    "status": "completed",
    "progress": 100,
    "cost": 0,
    "credits_cost": 0,
    "result": {
      "source_task_id": "task_...",
      "image_id": "6b78a7c3-5f9e-4a3d-a928-9e9b42113a3f",
      "image_url": "https://.../source.jpg",
      "from_cache": true,
      "cache_status": "hit",
      "objects": [{
        "index": 0,
        "name": "red sports car",
        "box_xyxy": [38.1, 689.8, 945.8, 1065.4],
        "score": 0.9765625,
        "mask_size": [1792, 1008],
        "mask_url": "",
        "mask_rle": { "size": [1792, 1008], "counts": "..." }
      }]
    }
  }
}
```

| Field | Description |
| - | - |
| `result.image_id` | Asset ID used by `region_edit` |
| `result.image_url` | HTTP(S) URL aligned with `image_id` |
| `objects[].index` | Original server index; preserve it when using `object_indices` |
| `objects[].box_xyxy` | Mask pixel box `[x1,y1,x2,y2]` |
| `objects[].score` | Detection confidence; may be `null` |
| `objects[].mask_size` | Always `[height,width]`; never hard-code dimensions |
| `objects[].mask_rle` | COCO compressed RLE for precise outlines |
| `objects[].mask_url` | Optional mask image URL; may be empty |

An object without a valid `mask_rle` or `mask_url` can only use approximate box editing.

## Decode `mask_rle`

`mask_rle.counts` is a COCO compressed-count string, not Base64 or zlib. It expands in column-major order; the first run is background, followed by alternating foreground and background runs.

The following TypeScript converts it into a browser-friendly row-major binary mask:

```ts theme={null}
export interface CocoRLE {
  size: [height: number, width: number];
  counts: string;
}

export interface BinaryMask {
  width: number;
  height: number;
  data: Uint8Array; // data[y * width + x]
}

function decodeCompressedCounts(counts: string): number[] {
  const runs: number[] = [];
  let cursor = 0;
  while (cursor < counts.length) {
    let value = 0;
    let shift = 0;
    let more = true;
    while (more) {
      if (cursor >= counts.length) throw new Error("Truncated COCO RLE counts");
      const current = counts.charCodeAt(cursor++) - 48;
      value |= (current & 0x1f) << shift;
      more = (current & 0x20) !== 0;
      shift += 5;
      if (!more && (current & 0x10) !== 0) value |= -1 << shift;
    }
    if (runs.length > 2) value += runs[runs.length - 2] ?? 0;
    if (value < 0) throw new Error(\`Invalid COCO RLE run: \${value}\`);
    runs.push(value);
  }
  return runs;
}

export function decodeCocoRLE(rle: CocoRLE): BinaryMask {
  const [height, width] = rle.size;
  if (!Number.isInteger(height) || !Number.isInteger(width) || height <= 0 || width <= 0) {
    throw new Error(\`Invalid mask size: \${JSON.stringify(rle.size)}\`);
  }
  const pixelCount = width * height;
  if (!Number.isSafeInteger(pixelCount) || pixelCount > 32_000_000) {
    throw new Error(\`Mask exceeds the frontend safety limit: \${pixelCount}\`);
  }
  if (!rle.counts) throw new Error("Missing COCO RLE counts");

  const data = new Uint8Array(pixelCount);
  const runs = decodeCompressedCounts(rle.counts);
  let position = 0;
  let foreground = false;
  for (const run of runs) {
    if (position + run > data.length) throw new Error("COCO RLE exceeds mask_size");
    if (foreground) {
      for (let offset = 0; offset < run; offset++) {
        const index = position + offset;
        const y = index % height;
        const x = (index - y) / height;
        data[y * width + x] = 1;
      }
    }
    position += run;
    foreground = !foreground;
  }
  if (position !== data.length) throw new Error("COCO RLE does not cover the mask");
  return { width, height, data };
}
```

Decode large masks in a Web Worker. Never send complete `mask_rle.counts` values to logs, analytics, URLs, or error reporting.

### Convert masks to precise selections

```ts theme={null}
interface SelectionBoundary { points: number[] }
interface SelectionRegion { outer: SelectionBoundary; holes?: SelectionBoundary[] }
```

Trace connected components and holes, simplify the contours, and normalize every point to `0–1`. Each ring needs at least 3 distinct points, must have non-zero area, and must not self-intersect. Keep at most 16 largest regions per layer and 400 points per ring.

<Warning>
  `mask_size` is `[height,width]` and uses source-mask coordinates, not CSS display coordinates. With `object-fit: contain`, subtract letterbox offsets, scale against the actual drawn area, and clamp the final values to `0–1`.
</Warning>

Reading source-image or `mask_url` pixels requires CORS. Set `crossOrigin = "anonymous"` before `src`, or fetch a Blob. Decoding `mask_rle` directly avoids this dependency.

## Edit a region: `region_edit`

### Request parameters

| Field | Type | Required | Description |
| - | - | :-: | - |
| `model` | string | ✅ | Fixed to `grok-imagine-2.0-ext` |
| `operation` | string | ✅ | `region_edit` |
| `nsfw_check` | boolean | — | Default: `false`.<br />`true`: use `omni-moderation-latest` to review the edit prompt and input image.<br />`false` or omitted: do not send a moderation request. |
| `image_id` | string | ✅ | Source asset ID; first use `segment` result `image_id`, then use the latest edit result |
| `prompt` | string | ✅ | Non-empty instruction describing the desired change |
| `selection_regions` | array | \* | Normalized `0–1` polygons with `outer` and optional `holes`; recommended |
| `boxes` | number\[]\[] | \* | Rectangles `[x1,y1,x2,y2]`; pixel boxes require `mask_size` |
| `object_indices` | integer\[] | \* | Original `objects[].index` values; approximate box editing only |
| `mask_size` | integer\[] | \* | Required for pixel boxes; `[height,width]` with positive integers |

At least one of `selection_regions`, `boxes`, or `object_indices` must be non-empty. The API accepts combinations, but the frontend should use one method per request.

<Warning>
  Do not send `billing_model_name`, `size`, `aspect_ratio`, `source_aspect_ratio`, `source_size`, or `image_urls`. Omit `n` or set it to `1`; omit `claim_asset` or set it to `false`; omit `response_format` or set it to `url`. Base64 output and `stream=true` are unsupported.
</Warning>

### Selection methods

| Method | Selection source | Precision | Recommended use |
| - | - | - | - |
| `selection_regions` | Frontend polygons | Exact, including holes | Production layer or brush editing |
| `boxes` | Frontend rectangles | Box approximation | Box tool or MVP |
| `object_indices` | Original segment indices | Box approximation | Quick integration testing |

<Tabs>
  <Tab title="Precise polygon">
    ```json theme={null}
    {
      "model": "grok-imagine-2.0-ext",
      "operation": "region_edit",
      "image_id": "<SOURCE_IMAGE_ID>",
      "prompt": "Change the selected car to bright red and preserve the rest",
      "selection_regions": [{
        "outer": { "points": [0.12, 0.20, 0.48, 0.20, 0.48, 0.61, 0.12, 0.61] },
        "holes": [{ "points": [0.30, 0.35, 0.38, 0.35, 0.38, 0.45, 0.30, 0.45] }]
      }]
    }
    ```

    `points` can be flat or nested pairs. Every value must be finite and within `0–1`; each ring needs at least 3 coordinate pairs.
  </Tab>

  <Tab title="Normalized box">
    ```json theme={null}
    {
      "model": "grok-imagine-2.0-ext",
      "operation": "region_edit",
      "image_id": "<SOURCE_IMAGE_ID>",
      "prompt": "Change the car inside the box to bright red",
      "boxes": [[0.04, 0.385, 0.938, 0.594]]
    }
    ```
  </Tab>

  <Tab title="Pixel box">
    ```json theme={null}
    {
      "model": "grok-imagine-2.0-ext",
      "operation": "region_edit",
      "image_id": "<SOURCE_IMAGE_ID>",
      "prompt": "Change the car inside the box to bright red",
      "boxes": [[40, 689.6, 945.9, 1064.4]],
      "mask_size": [1792, 1008]
    }
    ```
  </Tab>

  <Tab title="Object index">
    ```json theme={null}
    {
      "model": "grok-imagine-2.0-ext",
      "operation": "region_edit",
      "image_id": "<SOURCE_IMAGE_ID>",
      "prompt": "Change the selected car to bright red",
      "object_indices": [0]
    }
    ```

    Indices must come from the segment response for the same `image_id`. Do not replace them with indices from a filtered, sorted, or grouped frontend array.
  </Tab>
</Tabs>

### Completed response

```json theme={null}
{
  "code": 200,
  "data": {
    "id": "task_...",
    "status": "completed",
    "progress": 100,
    "cost": 0.016,
    "credits_cost": 0.16,
    "result": {
      "images": [{
        "url": ["https://.../result.jpg"],
        "image_ids": ["<NEW_IMAGE_ID>"],
        "items": [{
          "url": "https://.../result.jpg",
          "image_id": "<NEW_IMAGE_ID>",
          "source_image_id": "<SOURCE_IMAGE_ID>",
          "role": "region_edit"
        }],
        "expires_at": 1787040000
      }]
    }
  }
}
```

Prefer `result.images[0].items[0]`. For legacy responses, pair `url[0]` with `image_ids[0]` only when the arrays have equal lengths. Continue only after obtaining both an HTTP(S) URL and a new `image_id`.

Use `expires_at` as the source of truth for URL expiry; do not hard-code a number of hours. Download or persist assets needed for long-term display.

## Continuous editing

After an edit completes, update the displayed URL, the current asset ID, and the source task ID together, then clear old layers and polling state.

* Segment again: use this `region_edit` task ID as `source_task_id`
* Edit again: use the newly returned `image_id`
* Never pass `image_id` to `segment`, and never keep editing the previous image ID.

## Error handling

| HTTP / status | Common cause | Handling |
| - | - | - |
| 400 invalid source or operation | Wrong operation; both or neither source supplied; unusable task; invalid image URL; or `image_id/image_index` sent to segment | Choose exactly one valid source. For uploads, send one public HTTP(S) URL with `cache_only=true` |
| 400 invalid selection | Empty prompt, missing selection, invalid polygon, box, or object index | Validate the prompt and selection before submission |
| 400 unsupported option | Invalid `claim_asset`, `n`, output format, size, or stream mode | Remove unsupported fields and use URL output |
| 401 / 403 | Invalid key or missing model permission | Check the server-side key and account access |
| 402 | Insufficient balance for a paid edit | Ask the user to top up before retrying |
| 409 | Idempotency request is in progress, changed, or indeterminate | Follow the response; do not switch keys automatically |
| 429 / 5xx | Rate limit or temporary service failure | Honor `Retry-After` and retry with bounded backoff |
| failed / task\_failed | Asynchronous execution failed | Stop polling and show `data.error.message` |

## Billing

* `segment` is free and completes with `cost=0` and `credits_cost=0`, but still requires authentication and a valid source input.
* `region_edit` is paid. Use the completed task's `cost` and `credits_cost`; do not hard-code prices in the frontend.
* Never send the internal field `billing_model_name`.

## Frontend checklist

* Keep the API key only in the backend or BFF.
* Send one `segment` source: `source_task_id`, or `image_urls` containing one public URL. Do not send `image_id` or `image_index`.
* With `image_urls`, set `cache_only=true` and omit `cached_only` and `refresh`.
* Use segment's `image_id` for `region_edit` and provide at least one selection method.
* Use `selection_regions` for precise production editing; `object_indices` is only a box approximation.
* Always parse `mask_size` as `[height,width]` and account for display scaling and letterboxing.
* Reuse the original idempotency key for the same network retry and validate both the output URL and new `image_id`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.