Not accessible under automatic1111 API

[Graylington]

Is there any way to access this via the automatic1111 API? I've noticed other extensions on my list are accessible.

[graemeniedermayer]

I've managed to use this extension with the api.

in javascript the request looked like

depth_args ={
    'compute_device':0, 
    'model_type':1, 
    'net_width':512, 
    'net_height':512, 
    'match_size':true, 
    'invert_depth':true, 
    'boost':false, 
    'save_depth':true, 
    'show_depth':true, 
    'show_heat':false, 
    'combine_output':false, 
    'combine_output_axis':0, 
    'gen_stereo':false, 
    'gen_anaglyph':false, 
    'stereo_divergence':2.5, 
    'stereo_fill':0, 
    'stereo_balance':0, 
    'clipdepth':false, 
    'clipthreshold_far':1, 
    'clipthreshold_near':0.0, 
    'inpaint':false, 
    'inpaint_vids':false, 
    'background_removal_model':'u2net',
    'background_removal':true, 
    'pre_depth_background_removal':false, 
    'save_background_removal_masks':false,
     'gen_normal':false,
}

for the depth arguments and then

dic = {
        "init_images": [OrigImage],
        // 'mask': lastMask,
        'prompt': params.prompt,
        'negative_prompt': params.negative_prompt,
        'steps': params.steps,
        // 'inpainting_mask_invert':0,
        // 'inpainting_fill':1,
        'sampler_name': params.sampler_name, 
        'cfg_scale': params.cfg_scale, 
        'seed': seed, 
        'width': width,
        'height': height, 
        'denoising_strength': denoising_strength, 
        'script_name': scriptname,
        'script_args': Object.values(depth_args)
        // 'mask_blur': 2
    }
fetch(url, 
        {
            method: 'POST',
            body: JSON.stringify(dic), 
            headers: {
                'Content-Type': 'application/json'
              },
            mode:'cors',  
        }
    )

You might not need cors. It's mostly the same in python. There should be some way of making it more accessible to fastapi's listings.

[benedlore]

I've managed to use this extension with the api.

in javascript the request looked like

depth_args ={
    'compute_device':0, 
    'model_type':1, 
    'net_width':512, 
    'net_height':512, 
    'match_size':true, 
    'invert_depth':true, 
    'boost':false, 
    'save_depth':true, 
    'show_depth':true, 
    'show_heat':false, 
    'combine_output':false, 
    'combine_output_axis':0, 
    'gen_stereo':false, 
    'gen_anaglyph':false, 
    'stereo_divergence':2.5, 
    'stereo_fill':0, 
    'stereo_balance':0, 
    'clipdepth':false, 
    'clipthreshold_far':1, 
    'clipthreshold_near':0.0, 
    'inpaint':false, 
    'inpaint_vids':false, 
    'background_removal_model':'u2net',
    'background_removal':true, 
    'pre_depth_background_removal':false, 
    'save_background_removal_masks':false,
     'gen_normal':false,
}

for the depth arguments and then

dic = {
        "init_images": [OrigImage],
        // 'mask': lastMask,
        'prompt': params.prompt,
        'negative_prompt': params.negative_prompt,
        'steps': params.steps,
        // 'inpainting_mask_invert':0,
        // 'inpainting_fill':1,
        'sampler_name': params.sampler_name, 
        'cfg_scale': params.cfg_scale, 
        'seed': seed, 
        'width': width,
        'height': height, 
        'denoising_strength': denoising_strength, 
        'script_name': scriptname,
        'script_args': Object.values(depth_args)
        // 'mask_blur': 2
    }
fetch(url, 
        {
            method: 'POST',
            body: JSON.stringify(dic), 
            headers: {
                'Content-Type': 'application/json'
              },
            mode:'cors',  
        }
    )

Can you post the url endpoint you used with this payload? And what exactly is scriptname?

[graemeniedermayer]

Oh yes I missed that. At the time the value was scriptname = "DepthMap v0.3.8" now it'll be scriptname = "DepthMap v0.3.10"

I haven't tested this yet with v0.3.10

[thygate]

@graemeniedermayer Yes, the param count has changed too now, I have contemplated removing the version number from the scriptname for this reason, and only print it to console.

It also still has the gen_normal code I put in there for my own use that i forgot to remove before a commit at some point, so I could generate it all at once instead of having to make another api call and re-generate depth etc.. I will remove this in one of the future commits, perhaps you would like to make a MR to include it again at some point..

[thygate]

I still have not figured out how to setup the api to interace with the depth functions directly, instead of having to go through txt2img or img2img api with script_name ..

Here's the current parameter list : compute_device, model_type, net_width, net_height, match_size, boost, invert_depth, clipdepth, clipthreshold_far, clipthreshold_near, combine_output, combine_output_axis, save_depth, show_depth, show_heat, gen_stereo, stereo_modes, stereo_divergence, stereo_fill, stereo_balance, inpaint, inpaint_vids, background_removal, save_background_removal_masks, gen_normal, pre_depth_background_removal, background_removal_model, gen_mesh, mesh_occlude, mesh_spherical

[graemeniedermayer]

Yah! I was thinking about how update normal maps to include zoedepth (it works so well with faces), but it probably makes more sense to work on a MR so that I don't have to integrate each new update.

Also I was wondering if there's a reason for not passing the parameters a python dictionary/**kwargs. It would probably make API calls more stable (because the number of function parameters wouldn't change and you could re-order parameters), but there might be other ramifications I'm not thinking about.

[thygate]

I have not looked into it in detail, other extensions seem to be doing the same thing, but it is getting cumbersome to add/change parameters, and impossible to have a consistent api in the current state, yes. I would prefer an api endpoint to link straight to some depth functions, skip the whole img2img with script_name and args ..

What's your opinion on removing the version number from the script name ?

[graemeniedermayer]

I think it makes a lot of sense to move the version number to command line only.

A more direct api also makes a lot of sense. Especially with controlnet and other methods using depth maps as inputs, it feels clunky to call img2img with a lot of unused parameters.

[Straafe]

I still have not figured out how to setup the api to interace with the depth functions directly, instead of having to go through txt2img or img2img api with script_name ..

Looking at other extensions that provide their own unique/direct API's, it looks like they might just include a file defining those things, like api.py in the controlnet extension? I also noticed the dreambooth extension might have provided its own unique API's through A1111, so it might also provide an example but I no longer have it installed.

Edit: Here is the dreambooth extension's api script.

basic example

    @app.get("/myextension/myapi")
    async def get_some_data(
            expectedArg1: str = Query(description="Some data the user can provide when calling this", ),
            expectedArg1: str = Query("", description="Some data the user can provide when calling this", )
    ) -> JSONResponse:
        """
        This is my unique api
        """

        if there_was_an_error:
            return JSONResponse(status_code=422, content={"message": "There was an error."})

        return JSONResponse(content=data_user_wants)

[thygate]

Thanks for the tip, I will take a look at those when I find the time. There was little to no documentation or examples when this scripts started, there seem to be plenty of good examples now, so many in fact that I appreciate you naming a few relevant ones here.

[Straafe]

I don't use Python very often and move pretty slowly, but was able to set up a Hello World example in the text2video extension which also does not have any API access currently. Added api.py to its scripts directory:

import base64
import functools
import hashlib
import io
import json
import logging
import os
import shutil
import traceback
import zipfile

from PIL import Image
from fastapi import FastAPI, Response, Query, Body, Form, Header
from fastapi.encoders import jsonable_encoder
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse, StreamingResponse, FileResponse
from pydantic import BaseModel, Field
from starlette import status
from starlette.requests import Request

logger = logging.getLogger(__name__)

def t2v_api(_, app: FastAPI):
    logger.debug("Loading T2V API Endpoints.")
    @app.exception_handler(RequestValidationError)
    async def validation_exception_handler(request: Request, exc: RequestValidationError):
        return JSONResponse(
            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
            content=jsonable_encoder({"detail": exc.errors(), "body": exc.body}),
        )

    @app.get("/t2v/run")
    async def t2v_run(
            test: str = Query("", description="test", )
    ):
        """
        Run t2v over api
        @return:
        """
        return JSONResponse(content={"message": f"Hello! " + test})

try:
    import modules.script_callbacks as script_callbacks

    script_callbacks.on_app_started(t2v_api)
    logger.debug("SD-Webui API layer loaded")
except:
    logger.debug("Unable to import script callbacks")
    pass

Added a couple lines to the main extension script as well

def _initialize_api(self, app: FastAPI):
        return scripts.api.t2v_api(None, app)

And sure enough, the new api appears in the docs and runs