[BUG] 2 Instances of Go Service running, Job Queue fails

[lovgrandma]

I have an asynq job queue running on a PROD Go service that is functioning well in production and then I have a DEV Go service running on the same server. I have them pointing to different redis ports but it seems that both job queues fail when I have the PROD and DEV running at the same time. If I have one running at the same time its no issue. It is a video transcode process.

video:61kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown 424|tycoon-services-dev | Streaming map[format:map[bit_rate:1795157 duration:63.563500 filename:../tycoon-services-video-upload/0d66e52f99d04447b18977cb9390c633.mp4 format_long_name:QuickTime / MOV format_name:mov,mp4,m4a,3gp,3g2,mj2 nb_programs:0 nb_streams:1 probe_score:100 size:14263312 start_time:0.000000 tags:map[compatible_brands:iso8isommp41dashavc1cmfc creation_time:2022-12-02T03:51:02.000000Z major_brand:mp41 minor_version:0]] streams:[map[avg_frame_rate:30000/1001 bit_rate:268310 bits_per_raw_sample:8 chroma_location:left codec_long_name:H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 codec_name:h264 codec_tag:0x31637661 codec_tag_string:avc1 codec_time_base:1001/60000 codec_type:video coded_height:720 coded_width:1280 display_aspect_ratio:16:9 disposition:map[attached_pic:0 clean_effects:0 comment:0 default:1 dub:0 forced:0 hearing_impaired:0 karaoke:0 lyrics:0 original:0 timed_thumbnails:0 visual_impaired:0] duration:63.563500 duration_ts:1.906905e+06 has_b_frames:0 height:720 index:0 is_avc:true level:30 nal_length_size:4 pix_fmt:yuv420p profile:Constrained Baseline r_frame_rate:30000/1001 refs:1 sample_aspect_ratio:1:1 start_pts:0 start_time:0.000000 tags:map[creation_time:2022-12-02T03:51:02.000000Z encoder:AVC Coding handler_name:VideoHandler language:eng] time_base:1/30000 width:1280]]] 424|tycoon-services-dev | asynq: pid=28952 2023/02/13 21:09:43.421910 ERROR: recovering from panic. See the stack trace below for details: 424|tycoon-services-dev | goroutine 205 [running]: 424|tycoon-services-dev | runtime/debug.Stack() 424|tycoon-services-dev | /usr/lib/golang/src/runtime/debug/stack.go:24 +0x65 424|tycoon-services-dev | github.com/hibiken/asynq.(processor).perform.func1() 424|tycoon-services-dev | /root/go/pkg/mod/github.com/hibiken/asynq@v0.23.0/processor.go:416 +0x5d 424|tycoon-services-dev | panic({0x12c08c0, 0xc000640e10}) 424|tycoon-services-dev | /usr/lib/golang/src/runtime/panic.go:838 +0x207 424|tycoon-services-dev | tycoon.systems/tycoon-services/video/video_queue/transcode.PackageManifest(0xc0005e73b0, {0xc0005f5c00, 0x5, 0x8}, 0x1) 424|tycoon-services-dev | /home/ec2-user/tycoon-systems-servers/development/tycoon-services/video/video_queue/transcode/transcode.go:159 +0x1265 424|tycoon-services-dev | tycoon.systems/tycoon-services/video/video_queue.PerformVideoProcess(0xc0005e73b0) 424|tycoon-services-dev | /home/ec2-user/tycoon-systems-servers/development/tycoon-services/video/video_queue/video_queue.go:110 +0x139 424|tycoon-services-dev | tycoon.systems/tycoon-services/video/video_queue.HandleVideoProcessTask({0x899ce0?, 0xc00009aaa0?}, 0xc0004944b0) 424|tycoon-services-dev | /home/ec2-user/tycoon-systems-servers/development/tycoon-services/video/video_queue/video_queue.go:92 +0x172 424|tycoon-services-dev | github.com/hibiken/asynq.HandlerFunc.ProcessTask(0xc00009aaa0?, {0x16534b8?, 0xc000371080?}, 0xc0000ee340?) 424|tycoon-services-dev | /root/go/pkg/mod/github.com/hibiken/asynq@v0.23.0/server.go:577 +0x2f 424|tycoon-services-dev | github.com/hibiken/asynq.(ServeMux).ProcessTask(0x7fdcae9825b8?, {0x16534b8, 0xc000371080}, 0xc0004944b0?) 424|tycoon-services-dev | /root/go/pkg/mod/github.com/hibiken/asynq@v0.23.0/servemux.go:51 +0x46 424|tycoon-services-dev | github.com/hibiken/asynq.(processor).perform(0xc0004cef80?, {0x16534b8?, 0xc000371080?}, 0x0?) 424|tycoon-services-dev | /root/go/pkg/mod/github.com/hibiken/asynq@v0.23.0/processor.go:432 +0x82 424|tycoon-services-dev | github.com/hibiken/asynq.(processor).exec.func1.3() 424|tycoon-services-dev | /root/go/pkg/mod/github.com/hibiken/asynq@v0.23.0/processor.go:230 +0x1ca 424|tycoon-services-dev | created by github.com/hibiken/asynq.(*processor).exec.func1 424|tycoon-services-dev | /root/go/pkg/mod/github.com/hibiken/asynq@v0.23.0/processor.go:219 +0x265

To Reproduce Steps to reproduce the behavior (Code snippets if applicable):

1. Create a working job queue using basic docs for asynq
2. Make another copy of Go service
3. Point second Go Service to different running redis port
4. Run both
5. Attempt to process job
6. Should fail even though ports are different

Expected behavior Expect domains of Job Queue to be separated by Go instance and Redis port

Environment (please complete the following information): AWS EC2 t2.medium

The job queue is made to transcode videos but I use it for nearly everything else that needs to be in a queue process. My other option is to use another EC2 server but that is expensive.

[linhbkhn95]

Should double-check your code. I see a log error that your code has a panic error at

424|tycoon-services-dev | /home/ec2-user/tycoon-systems-servers/development/tycoon-services/video/video_queue/transcode/transcode.go:159 +0x1265

I think it is a nil pointer error. feel free to ping me at trinhducbaolinh@gmail.com for supporting

[lovgrandma]

False alarm, this was an internal issue.

yes a nil pointer. I put a video into my job queue that didnt have an audio track. Had to update logic to my server. Unrelated to asynq. Thanks again for the great tool. Expect a donation.

[linhbkhn95]

thank you for your response

[linhbkhn95]

can you mark the issue as closed status? @lovgrandma