search

isucon / isucon11-qualify

ISUCON11 予選 (ISUCONDITION)
https://isucon.net/archives/55821036.html
MIT License
166 stars 83 forks source link

[infra] 競技者インスタンスを立ち上げた時 isucondition.go が立ち上がってないことがある #1385

Closed ShotaKitazawa closed 3 years ago

ShotaKitazawa commented 3 years ago

After が効いてない？？

https://github.com/isucon/isucon11-qualify/blob/3ad5d9d66d0e59464d6656565abf9ceffc72d419/provisioning/ansible/roles/contestant/files/etc/systemd/system/isucondition.go.service#L1-L17

ShotaKitazawa commented 3 years ago

/var/lib/cloud/scripts/per-instance/generate-env.sh の実行が終わるよりも前に isucondition.go が上がってしまってる説

ShotaKitazawa commented 3 years ago

@takonomura さんより (cloud-init を使わずに) isucondition.xx.service の ExecStartPre に /home/isucon/env.sh の生成スクリプトを指定するでも良さそう

ただこの場合、 「env.sh の生成」というインフラ都合なものがより選手から見える位置に来るので混乱がありそう 今の所感は、あくまで↑の案は代替案くらい

Nagarei commented 3 years ago

afterは機能している

isucon@ip-192-168-0-71:~$ sudo systemctl status cloud-config.service
● cloud-config.service - Apply the settings specified in cloud-config
     Loaded: loaded (/lib/systemd/system/cloud-config.service; enabled; vendor preset: enabled)
     Active: active (exited) since Fri 2021-08-20 15:24:49 JST; 11min ago
    Process: 423 ExecStart=/usr/bin/cloud-init modules --mode=config (code=exited, status=0/SUCCESS)
   Main PID: 423 (code=exited, status=0/SUCCESS)
      Tasks: 0 (limit: 4454)
     Memory: 12.1M
     CGroup: /system.slice/cloud-config.service

Aug 20 15:24:46 ip-192-168-0-71 systemd[1]: Starting Apply the settings specified in cloud-config...
Aug 20 15:24:49 ip-192-168-0-71 cloud-init[515]: Cloud-init v. 21.2-3-g899bfaa9-0ubuntu2~20.04.1 running 'modules:config' at Fri, 20 Aug 2021 06:24:46 +0000. Up 25.24 seconds.
Aug 20 15:24:49 ip-192-168-0-71 systemd[1]: Finished Apply the settings specified in cloud-config.
isucon@ip-192-168-0-71:~$ sudo journalctl -u isucondition.go.service -n1000
-- Logs begin at Fri 2021-08-20 15:24:29 JST, end at Fri 2021-08-20 15:37:50 JST. --
Aug 20 15:24:50 ip-192-168-0-71 systemd[1]: isucondition.go.service: Failed to load environment files: No such file or directory
Aug 20 15:24:50 ip-192-168-0-71 systemd[1]: isucondition.go.service: Failed to run 'start' task: No such file or directory
Aug 20 15:24:50 ip-192-168-0-71 systemd[1]: isucondition.go.service: Failed with result 'resources'.
Aug 20 15:24:50 ip-192-168-0-71 systemd[1]: Failed to start isucondition.go.
Aug 20 15:24:51 ip-192-168-0-71 systemd[1]: isucondition.go.service: Scheduled restart job, restart counter is at 1.
Aug 20 15:24:51 ip-192-168-0-71 systemd[1]: Stopped isucondition.go.
Aug 20 15:24:51 ip-192-168-0-71 systemd[1]: isucondition.go.service: Failed to load environment files: No such file or directory
Aug 20 15:24:51 ip-192-168-0-71 systemd[1]: isucondition.go.service: Failed to run 'start' task: No such file or directory
...
Nagarei commented 3 years ago

ExecStartPre に↓のスクリプトを突っ込む

#!/bin/sh
FILE="/home/isucon/env.sh"

while [ ! -e $FILE ]
do
  echo "check env.sh...";
  sleep 1
done
Nagarei commented 3 years ago

cloud-initでsystemd enableを叩くと良さそう

sorah commented 3 years ago

cloud-init.target より後にすればよくない？

https://cloudinit.readthedocs.io/en/latest/topics/boot.html