xmake-io / xmake

🔥 A cross-platform build utility based on Lua
https://xmake.io
Apache License 2.0
9.84k stars 776 forks source link

在 package 里进行 os.cd('-') 有概率会导致主项目的 xmake.lua 被覆盖 #4390

Closed zeromake closed 10 months ago

zeromake commented 10 months ago

Xmake 版本

v2.8.5+20231107

操作系统版本和架构

macOS Monterey 12.6

描述问题

在使用 add_repositories('xrepo https://github.com/zeromake/xrepo.git') 添加自己的 repo 时发现的这个问题,我自己的一个 package 的 install 有一段这样的逻辑

on_install(function()
os.cd("src")
os.vrunv("python", {"script.py"})
os.cd("-")
end)

实际脚本

有比较小的概率会出现这段逻辑执行后,也就是 os.cd("-") 回到了主项目目录而不是这个 package cache 里的目录 然后就是主项目的 xmake.lua 被覆盖,并且会报错

error: fatal: repository 'projectdir/.xmake/linux/x86_64/repositories/xrepo' does not exist

实际出现地方

这里想问一下,os.cd 的对应到各个 package on_install 时是否有做隔离,没有的话我有一个猜测,下面是我加了 print 的正常过程(因为不论是加了 print 还是加 -vD 选项都会有导致该情况出现变少)

cd /Users/zero/project/ONScripter-zero=>/Users/zero/.xmake/cache/packages/2311/l/luajit/2023.09.25/source
cd /Users/zero/.xmake/cache/packages/2311/l/luajit/2023.09.25/source=>/Users/zero/.xmake/cache/packages/2311/l/luajit/2023.09.25/source/src
cd /Users/zero/.xmake/cache/packages/2311/l/luajit/2023.09.25/source/src=>/Users/zero/.xmake/cache/packages/2311/l/luajit/2023.09.25/source

猜测的情况,可能完全不一样

# 进入 luajit cache
cd /Users/zero/project/ONScripter-zero=>/Users/zero/.xmake/cache/packages/2311/l/luajit/2023.09.25/source
# cd 到 src
cd /Users/zero/.xmake/cache/packages/2311/l/luajit/2023.09.25/source=>/Users/zero/.xmake/cache/packages/2311/l/luajit/2023.09.25/source/src
# os.vrunv 的阻塞时候有其它包可以编译,切换出去
cd /Users/zero/.xmake/cache/packages/2311/l/luajit/2023.09.25/source/src=>/Users/zero/project/ONScripter-zero
cd /Users/zero/project/ONScripter-zero=>/Users/zero/.xmake/cache/packages/2311/f/freetype/2.13.2/source
# 其它包释放并回到 luajit cache 上次目录
cd /Users/zero/.xmake/cache/packages/2311/f/freetype/2.13.2/source=>/Users/zero/project/ONScripter-zero
cd /Users/zero/project/ONScripter-zero=>/Users/zero/.xmake/cache/packages/2311/l/luajit/2023.09.25/source/src
# 这个时候 os.cd('-') 了
cd /Users/zero/.xmake/cache/packages/2311/l/luajit/2023.09.25/source/src=>/Users/zero/project/ONScripter-zero

期待的结果

可以正常的在 on_install 里随意 os.cd

工程配置

add_rules("mode.debug", "mode.release")
add_repositories('xrepo https://github.com/zeromake/xrepo.git')

local deps = {
    "zlib",
    "bzip2",
    "jpeg",
    "png",
    "webp",
    "sdl2_mixer",
    "brotli",
    "ghc_filesystem",
    "luajit",
    "stb",
}
for _, dep_name in ipairs(deps) do
    add_requires(dep_name)
end
target("demo")
    for _, dep_name in ipairs(deps) do
        add_packages(dep_name)
    end
    add_files("demo.c")

附加信息和错误日志

以上仅为猜测情况,这里提供一下出现的概率比较高的条件

我这边本地也会搞个脚本循环不停的去尝试重现

Issues-translate-bot commented 10 months ago

Bot detected the issue body's language is not English, translate it automatically.


Title: Performing os.cd('-') in package may cause xmake.lua of the main project to be overwritten.

waruqi commented 10 months ago

安装包是并行装的,理论上我内部做了 协程 task 的 curdir 隔离,但是也许有bug 导致偶尔泄漏了 也是有可能的,这个就不太好调了。。

你可以尽可能避免 cd ,尤其是 cd 之间执行 os.runv 这会导致进程触发 Wait 切到其他协程。。

可以改用 os.vrunv("python", {"script.py"}, {curdir = "src"}) 直接切 curdir

zeromake commented 10 months ago

@waruqi 好的,不过我猜测的情况不是 curdir 的泄露,而是恢复到之前协程时 os._PREDIR 被覆盖了,curdir 做了隔离,predir 有做吗?

waruqi commented 10 months ago

哦 ,你用的 cd("-") 啊。。这个我自己都不怎么用,目前这个是不支持隔离的,属于很早期的设计,那个时候还没并行调度支持,要支持这个,估计要改点东西

建议改成下面的形式

local oldir = os.cd("xxx")
os.vrunv()
os.cd(oldir)
zeromake commented 10 months ago

@waruqi 那应该是没有 curdir 的协程泄露,我自己改一下支持就行了,这个 issues 可以关了

Issues-translate-bot commented 10 months ago

Bot detected the issue body's language is not English, translate it automatically.


The installation package is installed in parallel. Theoretically, I have implemented curdir isolation for coroutine tasks internally, but there may be bugs that cause occasional leaks. This is not easy to adjust. .

You can avoid cd as much as possible, especially executing os.runv between cd which will cause the process to trigger Wait and switch to other coroutines. .

You can instead use os.vrunv("python", {"script.py"}, {curdir = "src"}) to cut curdir directly

Issues-translate-bot commented 10 months ago

Bot detected the issue body's language is not English, translate it automatically.


Oh, you used cd("-"). . I don’t use this myself very much. Currently, this does not support isolation. It is a very early design. At that time, there was no support for parallel scheduling.

It is recommended to change it to the following form:

local oldir = os.cd("xxx")
os.vrunv()
os.cd(oldir)
Issues-translate-bot commented 10 months ago

Bot detected the issue body's language is not English, translate it automatically.


@waruqi Okay, but my guess is not that curdir is leaked, but that os._PREDIR is overwritten when restoring to the previous coroutine. Curdir has been isolated. Has predir been done?

Issues-translate-bot commented 10 months ago

Bot detected the issue body's language is not English, translate it automatically.


@waruqi That should be a coroutine leak without curdir. I can just change it myself to support it. This issue can be closed.