Pipy Repo REST API didn't work

[Hou-Xiaoxuan]

Try REST API in https://flomesh.io/pipy/docs/zh/operating/repo/3-api, the interface behavior is inconsistent with expectations.

Repeat method：

[I] user@mac ~> # a new start pipy
[I] user@mac ~> curl http://localhost:6060/api/v1/repo
[I] user@mac ~> curl -X POST http://localhost:6060/api/v1/repo/hello
[I] user@mac ~> curl http://localhost:6060/api/v1/repo
/hello⏎                                                                         
[I] user@mac ~> curl -s http://localhost:6060/api/v1/repo/hello
{"version":"0","path":"/hello","main":"/main.js","files":["/main.js"],"editFiles":[],"erasedFiles":[],"baseFiles":[],"derived":[],"instances":{}}⏎              
[I] user@mac ~> curl -s http://localhost:6060/api/v1/repo/hello/main.js
((
  // Global variables go here, e.g.:
  // config = pipy.solve('config.js'),

) => pipy({
  // Context variables go here, e.g.:
  // _target: null,

})

  // Pipeline layouts go here, e.g.:
  .listen(80)
  .dump()
  .dummy()

)()
[I] user@mac ~> curl -X POST http://localhost:6060/api/v1/repo/hello/main.js --data """pipy()
                             .listen(8080)
                             .serveHTTP(
                               new Message('Hi!')
                             )"""
[I] user@mac ~> curl -s http://localhost:6060/api/v1/repo/hello/main.js
((
  // Global variables go here, e.g.:
  // config = pipy.solve('config.js'),

) => pipy({
  // Context variables go here, e.g.:
  // _target: null,

})

  // Pipeline layouts go here, e.g.:
  .listen(80)
  .dump()
  .dummy()

)()
[I] user@mac ~> # main.js didn't change
[I] user@mac ~> curl http://localhost:6060/api/v1/repo
/hello
/hello/main.js⏎                                                                 
[I] user@mac ~> curl -s http://localhost:6060/api/v1/repo/hello
{"version":"0","path":"/hello","main":"/main.js","files":["/main.js"],"editFiles":[],"erasedFiles":[],"baseFiles":[],"derived":[],"instances":{}}⏎              
[I] user@mac ~> # didn't have `edit files`
[I] user@mac ~> 

The update file interface appears to be recognized as the new repo interface。

Pipy version：

git log -1
commit 893fa2de9b3546c2f53b39bd2ec5f50e2503b597 (HEAD -> main)
Author: pajama-coder <pajamacoder@flomesh.io>
Date:   Wed Jun 5 13:30:22 2024 +0800

    [api] Use a flush in print() and println()

[ethinx]

@Hou-Xiaoxuan Sorry for the confusion, and the doc is not update to date.

The POST method to the /api/v1/repo/<codebase name> is for codebase creation.

For file operation, the API is /api/v1/repo-files/<codebase name>/path/to/file

Here is a sample script for codebase initialization, for your reference:

#!/bin/bash
# ./init-repo.sh http://localhost:6060/repo/demo path/to/local/codebase/dir/

REPO_URI=${1:-http://localhost:6060/repo/demo}
REPO_PATH=${2%%/}

API_PATH=${REPO_URI/\/repo\//\/api\/v1\/repo\/}

[ ! -d "$REPO_PATH" ] && exit 1

# delete repo
curl -X DELETE ${API_PATH}

# create repo
curl -X POST ${API_PATH}

# delete file
curl -X DELETE ${API_PATH/repo/repo-files}/main.js

find $REPO_PATH -type f | egrep -v ".*\.sh|.*\.py" | while read line;
do
        echo "Push file: ${line##$REPO_PATH/}"
        curl -X POST ${API_PATH/repo/repo-files}/${line##$REPO_PATH/} --data-binary "@$line"
done

# set the main entry of the codebase
curl -X PATCH ${API_PATH} --data '{"main":"/main.js"}'

# release the changes
curl -X PATCH ${API_PATH} --data '{"version": '$(date +%s)'}'

[Hou-Xiaoxuan]

I found the API related code at https://github.com/flomesh-io/pipy/blob/main/src/admin-service.cpp. Maybe I can view the code for information about the API.

BTW, I wonder if there is an API to let Pipy program exit? ( not the running repo but all pipy admin server) @ethinx

[ethinx]

@Hou-Xiaoxuan HTTP API? I don't think we have such kind of API, maybe you could elaborate on the senario...

[Hou-Xiaoxuan]

@Hou-Xiaoxuan HTTP API? I don't think we have such kind of API, maybe you could elaborate on the senario... HTTP API？我认为我们没有这样的 API，也许你可以详细说明一下情况......

we're working on wrap pipy as a rust crate to to simplify the use of pipy/ztm in rust, in this repo. Now, we run pipy as a rust thread like:

#[link(name = "pipy", kind = "dylib")]
extern "C" {
    pub fn pipy_main(argc: c_int, argv: *const *const c_char) -> c_int;
}
/// start pipy in repo mod with given port, default port is 6060
/// **Note**: If you invoke this function, you must call `libc::exit(0)` to exit the process finally,
/// otherwise the `pipy-main` thread will report a `panic!` when the process exits.
pub fn start_pipy_repo(port: Option<u16>) {
    thread::spawn(move || {
        let mut args: Vec<CString> = vec![];
        args.push(CString::new("pipy-rs").unwrap());
        args.push(CString::new(format!("--admin-port={}", port.unwrap_or(6060))).unwrap());
        let c_args: Vec<*const c_char> = args
            .iter()
            .map(|arg| <CString as Clone>::clone(&arg).into_raw() as *const c_char)
            .collect();

        unsafe {
            pipy_main(c_args.len() as c_int, c_args.as_ptr());
        }
    });
    thread::sleep(std::time::Duration::from_secs(1)); // wait for pipy to start
}

But we just can't terminate the pipy program appropriately because rust doesn't allow you to kill a thread directly. So I wish there was some way to stop pipy's main function from running.

Related works: https://github.com/flomesh-io/ztm/issues/9

[ethinx]

@pajama-coder do you have any suggestion for the situation?

[pajama-coder]

@Hou-Xiaoxuan I've added an exported function void pipy_exit(int force) in shared lib mode in ad274d972f3f6bf1310bbc924870b29d497aefd9. When force == 0, Pipy will terminate after draining all workloads. When force != 0, Pipy shutdown all connections and quit immediately. Please give it a try. Thanks.

[keveinliu]

@Hou-Xiaoxuan Hi， have you tried that solution in last comment?

[Hou-Xiaoxuan]

@Hou-Xiaoxuan Hi， have you tried that solution in last comment?

It works well in worker mode, but didn't exit in repo mode.