Jonathan is not thread-safe

[fukamachi]

While I'm looking into https://github.com/fukamachi/psychiq/issues/1, I found that jojo:to-json may be not thread-safe.

Sometimes it says like the following errors:

The bounding indices 729 and 440 are bad for a sequence of length 440.
See also:
  The ANSI Standard, Glossary entry for "bounding index designator"
  The ANSI Standard, writeup for Issue SUBSEQ-OUT-OF-BOUNDS:IS-AN-ERROR

And the output string from jojo:to-json is sometimes incomplete or contains garbages like:

"{{\"retry_count\":0,\"retry_\"of{ti\"led_at\":0,1481507440,\"failed_at\"\"queue\":\"de\"auet\",_cou14\":507440,\"error_class\"\"queue\"COMM\"ONdLIf0lSI\"M,PLE-ERROR\",\"\"failed_ats\"or:_messaCOe\"ON-\"ISPo:m:eSI PLhEi-E RRap1481ed\"440,\"error_messa\"queu\"::\"mOR\"Es:defahltppKERne\"d\",\"error\"a\"gl\"as\"\":[CWORKER::MY-WORKE1\",ER],R\",\"argsj\":\"er\"xeeuh54pm4id\"\"14m],shit ha\"cd_at\":\"jid\":\"xeeh54pm4id\",\"class\":\"WORK\"cr1at,d_at\":\",\"enqueue\"args\":1481507440,\"141]}ue07704"

Here's a reproducible code:

(in-package :cl-user)
(ql:quickload '(:legion :jonathan))

(defparameter *failed-count* 0)

(defvar *output-lock* (bt:make-lock))

(defparameter *cluster*
  (legion:make-cluster 30
                       (let ((out *standard-output*))
                         (lambda (worker)
                           (multiple-value-bind (val existsp)
                               (legion:next-job worker)
                             (handler-case
                                 (assert (string= "{\"retry_count\":0,\"failed_at\":1481507440,\"queue\":\"default\",\"error_class\":\"COMMON-LISP::SIMPLE-ERROR\",\"error_message\":\"Some shit happened\",\"class\":\"WORKER::MY-WORKER\",\"args\":[140],\"jid\":\"xeeuh54pm4id\",\"created_at\":1481507440,\"enqueued_at\":1481507704}"
                                                  (jojo:to-json val :from :alist)))
                               (error (e)
                                 (incf *failed-count*)
                                 (bt:with-lock-held (*output-lock*)
                                   (format out "~2&---------- ERROR ----------~%~A~%" e)))))))))

(legion:start-cluster *cluster*)

(dotimes (i 300)
  (legion:add-job *cluster* '(("retry_count" . 0)
                              ("failed_at" . 1481507440)
                              ("queue" . "default")
                              ("error_class" . "COMMON-LISP::SIMPLE-ERROR")
                              ("error_message" . "Some shit happened")
                              ("class" . "WORKER::MY-WORKER")
                              ("args" 140)
                              ("jid" . "xeeuh54pm4id")
                              ("created_at" . 1481507440)
                              ("enqueued_at" . 1481507704))))

[fukamachi]

Do you have any ideas to fix this?

I've fixed the issue of Psychiq with a dirty way, however, the situation may be happened in other environments. For instance, Woo's workers are threads and it would be a problem if the app uses jojo:to-json.

[rudolph-miller]

@fukamachi Naoshimashita! Arigato-gozaimasu!

[fukamachi]

Cool! Arigato!