search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18.34k stars 1.67k forks source link

[Support]: Cameras frozen after some time #4146

Closed psenakv closed 1 year ago

psenakv commented 1 year ago

Describe the problem you are having

After restarting Frigate (from UI or with docker compose down/docker compose up), everything works great. After some time (30 minut - 8 hours), all cameras froze on the same time (+-1 s). Frigate keeps running, the UI is visible, HA integration works, but all cameras show the frozen frame and recording doesn't work. After restart, all works great again, but it will froze again in some time. It did this on lower version (I think 0.10.?), so I upgraded it, but nothing changed. I use this as security system, so cameras freezing without Frigate crashing (which I can detect) is big problem for me.

Version

0.11.1-2EADA21

Frigate config file

mqtt:
  host: <HOST>
  user: <USER>
  password: <PASSWORD>
detect:
  fps: 5
  enabled: True
objects:
  track:
    - person
    - car
    - motorcycle
    - cat
    - broccoli
detectors:
  coral:
    type: edgetpu
    device: usb
record:
  enabled: True
  expire_interval: 60
  retain:
    days: 2
    mode: all
  events:
    pre_capture: 10
    post_capture: 10
    objects:
      - person
      - car
      - motorcycle
    retain:
      default: 9
      mode: motion
cameras:
  camera_1:
    ffmpeg:
      inputs:
        - path: <camera_url>
          roles:
            - detect
            - rtmp
            - record
  camera_2:
      ffmpeg:
        inputs:
          - path: <camera_url>
            roles:
              - detect
              - rtmp
              - record
  camera_3:
      ffmpeg:
        inputs:
          - path: <camera_url>
            roles:
              - detect
              - rtmp
              - record
  camera_4:
      ffmpeg:
        inputs:
          - path: <camera_url>
            roles:
              - detect
              - rtmp
              - record
  camera_5:
      ffmpeg:
        input_args: -an
        inputs:
          - path: <camera_url>
            roles:
              - rtmp
  camera_6:
      ffmpeg:
        inputs:
          - path: <camera_url>
            roles:
              - detect
              - rtmp
              - record
      detect:
        width: 1440
        height: 2560

Relevant log output

[2022-10-22 23:26:16] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:34168]
[2022-10-22 23:28:18] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:52140]
[2022-10-22 23:28:20] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:43550]
Fatal Python error: Fatal Python error: Bus errorBus error

Thread 0xThread 0x00007fdbc42b670000007fdbc42b6700 (most recent call first):
 (most recent call first):
  File   File ""//uussrr//lliibb//ppyytthhoonn33..99//tthhrreeaaddiinngg..ppyy"", line , line 312 in 312w in waaiitt

  File "/usr/lib/python3.9/multiprocessing/queues.py", line 233 in _feed
  File "/usr/lib/python3.9/threading.py", line 892 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File   File "/"u/surs/rl/ilbi/bp/yptyhtohno3n.3.99//tmhurletaidpirnogc.epsys"i, line n912g in /_qbuoeoutesst.rpayp"
, line
233Current thread 0x in 00007fdbc8ab7700_ (most recent call first):
f  File e"e/do
p  File t"/f/ruisgra/tlei/bf/rpiygtahtoen/3v.i9d/etoh.rpeya"d, line i168n in gc.appyt"u, line r892e in _rfurna
m  File e"s/
u  File s"r//olpitb//fpryitghaotne3/.f9r/itgharteea/dviindg.peyo"., line p954y in ", line _306b in oroutns
t  File r"a/pu_sirn/nleirb
/  File p"y/tuhsorn/3l.i9b//tphyrtehaodni3n.g9./ptyh"r, line e954a in d_ibnogo.tpsyt"r, line a912p in __ibnonoetrs
t  File r"a/pu
sr
Current thread 0x/l00007fdbc8ab7700i (most recent call first):
b  File /"p/yotphto/nf3r.i9g/atther/efardiignagt.ep/yv"i, line d912e in o_.bpoyo"t, line s168t in racpa
p
tThread 0xu00007fdbcb2b8700r (most recent call first):
e_  File f"r/aumsers/
l  File i"b//oppytt/hforni3g.a9t/em/uflrtiigpartoec/evsisdienog./psyy"n, line c306h in rrounni
z  File e"./puy"sr, line /261l in iwba/ipty
t  File h"o/nu3s.r9//ltibh/rpeyatdhionng3..p9y/"m, line u954l in t_ibporootcsetsrsaipn_gi/nsnyenrc
h  File r"on/iuzser./plyi"b, line /349p in ywtahiotn
3  File ."9//otphtr/efardiignagt.ep/yf"r, line i912g in a_tbeo/ovtisdteroa.pp
y
", line Thread 0x23200007fdbcb2b8700 in  (most recent call first):
r  File u"n/
u  File s"r//ulsirb//lpiybt/hpoynt3h.o9n/3m.u9l/ttihprreoacdeisnsgi.npgy/"s, line y954n in c_hbrooontiszter.appy_"i, line n261n in ewra
i  File t"
/  File u"s/ru/slri/bl/ipby/tphyotnh3o.n93/.t9h/rmeualdtiinpgr.opcye", line s912s in i_nbgo/ostynscthrraopn
i
zThread 0xe00007fdbcbab9700. (most recent call first):
py"  File , line "349/ in owpati/tf
ri  File g"a/toep/tf/rfirgiagtaet/el/forgi.gpayt"e, line /64v in irdueno
.  File p"y/"u, line sr232/ in lriubn/
p  File y"t/huosnr3/.l9i/bt/hpryetahdoinn3g..9p/yt"h, line r954e in a_dbionogt.sptyr"a, line p954_ in i_nbnoeort
s  File t"r/aups_ri/nlniebr/
p  File y"t/huosnr3/.l9i/tbh/rpeyatdhionng3..p9y/"t, line h912r in e_abdoiontgs.tprya"p, line
912
 in Thread 0x_00007fdbd5dc0740b (most recent call first):
o  File o"t/sutsrra/pl
i
bThread 0x/00007fdbcbab9700p (most recent call first):
yth  File o"n/3o.p9t//tfhrriegaadtien/gf.rpiyg"a, line t1049e in /_lwaoigt._pfyo"r, line _64t in srtautne
_  File l"o/cuksr/lib/python3.9/
t  File h"r/euasdri/nlgi.bp/yp"y, line th954o in n_3b.o9o/ttshtrreaapd_iinngn.epry
"  File , line "1033/ in ujsori/nl
ib/  File p"y/tohpotn/3f.r9i/gtahtree/afdriingga.tpey/"v, line i912d in e_ob.opoyt"s, line t337r in acpa
p
tThread 0xu00007fdbd5dc0740re (most recent call first):
_  File c"a/muesrra/
lib  File /"p/yutshro/nl3i.b9//ptyhtrheoand3i.n9g/.mpuyl"t, line i1049p in r_owcaeistsi_nfgo/rp_rtosctaetses_.lpoyc"k, line
108 in   File r"u/nu
sr/  File l"i/ubs/rp/yltihbo/np3y.t9h/otnh3r.e9a/dmiunlgt.ippyr"o, line c1033e in sjsoiinng
/p  File r"o/coepsts/.fpryi"g, line a315t in e_/bforoitgsattrea/pv
id  File e"o/.upsyr"/l, line i337b in /cpayptthuorne3_.c9a/mmeurlat
ip  File r"o/cuessrs/ilnigb//ppoyptehno_nf3o.r9k/.mpuyl"t, line i71p in r_olcaeusnscihn
g/p  File r"o/cuessrs/.lpiyb"/, line p108y in trhuonn3
.9  File /"m/uulstri/plriobc/epsystihnogn/3po.p9e/nm_uflotrikp.rpoyc"e, line s19s in i_n_gi/npirto_c_e
ss.  File p"y/"us, line r315/ in l_ibbo/otpsyttrhaopn
3.9  File /"m/uulstri/plirbo/cpeystshionng3/.c9o/nmtuelxtti.ppryo"c, line e277s in s_iPnogp/epno
p  File e"n/_ufsorr/kl.ipby/"py, line t71h in o_nla3u.n9c/hm
ulti  File p"/ruoscre/slsiibn/gp/cyotnhtoenx3t..9p/ym"u, line l224t in i_pProopceens
s  File i"n/gu/spro/pleinb_/fpoyrtkh.opny3".9, line /19m in u_l_itniiptr_o_c
ess  File i"n/gu/sprr/olciebs/sp.yptyh"o, line n1213 in .s9t/amrutl
tipr  File o"c/eospsti/nfgr/icgoanttexe/frtig.aptye"/, line ap277p in ._pPyo"p, line e286n in
s  File t"a/ruts_rc/almiebr/ap_yctahpotnu3r.e9_/pmruolcteispsreos
ces  File s"i/nogp/tc/ofnrtiegxatt.ep/yf"r, line i224ga in t_eP/oappepn.
p  File y""/, line u364s in rs/tlairbt/
python3  File ."9//omputl/tfirpirgoactees/sfirnigg/aptreo/c_e_smsa.ipny_"_, line .121p in ys"t, line a16r in t<
modu  File l"e/>o
pt/fr  File i"g/autser//flriibg/aptyet/haopnp3..p9y/"r, line u286n in psyt.apryt"_, line c87a in m_erruan__ccaopdteu
re_pr  File o"c/eusssres/
lib  File /"p/yotphto/nf3r.i9g/artuen/pfyr.ipgya"t, line e197/ in a_prpu.np_ym"od, line u364l in es_taasr_tm
ain
  File "/opt/frigate/frigate/__main__.py", line 16 in <module>
  File "/usr/lib/python3.9/runpy.py", line 87 in _run_code
  File "/usr/lib/python3.9/runpy.py", line 197 in _run_module_as_main
Fatal Python error: Bus error

Thread 0x00007fdbc42b6700 (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 312 in wait
  File "/usr/lib/python3.9/multiprocessing/queues.py", line 233 in _feed
  File "/usr/lib/python3.9/threading.py", line 892 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Current thread 0x00007fdbc8ab7700 (most recent call first):
  File "/opt/frigate/frigate/video.py", line 168 in capture_frames
  File "/opt/frigate/frigate/video.py", line 306 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Thread 0x00007fdbcb2b8700 (most recent call first):
  File "/usr/lib/python3.9/multiprocessing/synchronize.py", line 261 in wait
  File "/usr/lib/python3.9/multiprocessing/synchronize.py", line 349 in wait
  File "/opt/frigate/frigate/video.py", line 232 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Thread 0x00007fdbcbab9700 (most recent call first):
  File "/opt/frigate/frigate/log.py", line 64 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Thread 0x00007fdbd5dc0740 (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 1049 in _wait_for_tstate_lock
  File "/usr/lib/python3.9/threading.py", line 1033 in join
  File "/opt/frigate/frigate/video.py", line 337 in capture_camera
  File "/usr/lib/python3.9/multiprocessing/process.py", line 108 in run
  File "/usr/lib/python3.9/multiprocessing/process.py", line 315 in _bootstrap
  File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 71 in _launch
  File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 19 in __init__
  File "/usr/lib/python3.9/multiprocessing/context.py", line 277 in _Popen
  File "/usr/lib/python3.9/multiprocessing/context.py", line 224 in _Popen
  File "/usr/lib/python3.9/multiprocessing/process.py", line 121 in start
  File "/opt/frigate/frigate/app.py", line 286 in start_camera_capture_processes
  File "/opt/frigate/frigate/app.py", line 364 in start
  File "/opt/frigate/frigate/__main__.py", line 16 in <module>
  File "/usr/lib/python3.9/runpy.py", line 87 in _run_code
  File "/usr/lib/python3.9/runpy.py", line 197 in _run_module_as_main
Fatal Python error: Bus error

Thread 0x00007fdbc42b6700 (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 312 in wait
  File "/usr/lib/python3.9/multiprocessing/queues.py", line 233 in _feed
  File "/usr/lib/python3.9/threading.py", line 892 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Current thread 0x00007fdbc8ab7700 (most recent call first):
  File "/opt/frigate/frigate/video.py", line 168 in capture_frames
  File "/opt/frigate/frigate/video.py", line 306 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Thread 0x00007fdbcb2b8700 (most recent call first):
  File "/usr/lib/python3.9/multiprocessing/synchronize.py", line 261 in wait
  File "/usr/lib/python3.9/multiprocessing/synchronize.py", line 349 in wait
  File "/opt/frigate/frigate/video.py", line 232 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Thread 0x00007fdbcbab9700 (most recent call first):
  File "/opt/frigate/frigate/log.py", line 64 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Thread 0x00007fdbd5dc0740 (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 1049 in _wait_for_tstate_lock
  File "/usr/lib/python3.9/threading.py", line 1033 in join
  File "/opt/frigate/frigate/video.py", line 337 in capture_camera
  File "/usr/lib/python3.9/multiprocessing/process.py", line 108 in run
  File "/usr/lib/python3.9/multiprocessing/process.py", line 315 in _bootstrap
  File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 71 in _launch
  File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 19 in __init__
  File "/usr/lib/python3.9/multiprocessing/context.py", line 277 in _Popen
  File "/usr/lib/python3.9/multiprocessing/context.py", line 224 in _Popen
  File "/usr/lib/python3.9/multiprocessing/process.py", line 121 in start
  File "/opt/frigate/frigate/app.py", line 286 in start_camera_capture_processes
  File "/opt/frigate/frigate/app.py", line 364 in start
  File "/opt/frigate/frigate/__main__.py", line 16 in <module>
  File "/usr/lib/python3.9/runpy.py", line 87 in _run_code
  File "/usr/lib/python3.9/runpy.py", line 197 in _run_module_as_main
Fatal Python error: Bus error

Thread 0x00007fdbc42b6700 (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 312 in wait
  File "/usr/lib/python3.9/multiprocessing/queues.py", line 233 in _feed
  File "/usr/lib/python3.9/threading.py", line 892 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Current thread 0x00007fdbc8ab7700 (most recent call first):
  File "/opt/frigate/frigate/video.py", line 168 in capture_frames
  File "/opt/frigate/frigate/video.py", line 306 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Thread 0x00007fdbcb2b8700 (most recent call first):
  File "/usr/lib/python3.9/multiprocessing/synchronize.py", line 261 in wait
  File "/usr/lib/python3.9/multiprocessing/synchronize.py", line 349 in wait
  File "/opt/frigate/frigate/video.py", line 232 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Thread 0x00007fdbcbab9700 (most recent call first):
  File "/opt/frigate/frigate/log.py", line 64 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Thread 0x00007fdbd5dc0740 (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 1049 in _wait_for_tstate_lock
  File "/usr/lib/python3.9/threading.py", line 1033 in join
  File "/opt/frigate/frigate/video.py", line 337 in capture_camera
  File "/usr/lib/python3.9/multiprocessing/process.py", line 108 in run
  File "/usr/lib/python3.9/multiprocessing/process.py", line 315 in _bootstrap
  File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 71 in _launch
  File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 19 in __init__
  File "/usr/lib/python3.9/multiprocessing/context.py", line 277 in _Popen
  File "/usr/lib/python3.9/multiprocessing/context.py", line 224 in _Popen
  File "/usr/lib/python3.9/multiprocessing/process.py", line 121 in start
  File "/opt/frigate/frigate/app.py", line 286 in start_camera_capture_processes
  File "/opt/frigate/frigate/app.py", line 364 in start
  File "/opt/frigate/frigate/__main__.py", line 16 in <module>
  File "/usr/lib/python3.9/runpy.py", line 87 in _run_code
  File "/usr/lib/python3.9/runpy.py", line 197 in _run_module_as_main
Fatal Python error: Bus error

Thread 0x00007fdbc42b6700 (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 312 in wait
  File "/usr/lib/python3.9/multiprocessing/queues.py", line 233 in _feed
  File "/usr/lib/python3.9/threading.py", line 892 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Current thread 0x00007fdbc8ab7700 (most recent call first):
  File "/opt/frigate/frigate/video.py", line 168 in capture_frames
  File "/opt/frigate/frigate/video.py", line 306 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Thread 0x00007fdbcb2b8700 (most recent call first):
  File "/usr/lib/python3.9/multiprocessing/synchronize.py", line 261 in wait
  File "/usr/lib/python3.9/multiprocessing/synchronize.py", line 349 in wait
  File "/opt/frigate/frigate/video.py", line 232 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Thread 0x00007fdbcbab9700 (most recent call first):
  File "/opt/frigate/frigate/log.py", line 64 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap

Thread 0x00007fdbd5dc0740 (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 1049 in _wait_for_tstate_lock
  File "/usr/lib/python3.9/threading.py", line 1033 in join
  File "/opt/frigate/frigate/video.py", line 337 in capture_camera
  File "/usr/lib/python3.9/multiprocessing/process.py", line 108 in run
  File "/usr/lib/python3.9/multiprocessing/process.py", line 315 in _bootstrap
  File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 71 in _launch
  File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 19 in __init__
  File "/usr/lib/python3.9/multiprocessing/context.py", line 277 in _Popen
  File "/usr/lib/python3.9/multiprocessing/context.py", line 224 in _Popen
  File "/usr/lib/python3.9/multiprocessing/process.py", line 121 in start
  File "/opt/frigate/frigate/app.py", line 286 in start_camera_capture_processes
  File "/opt/frigate/frigate/app.py", line 364 in start
  File "/opt/frigate/frigate/__main__.py", line 16 in <module>
  File "/usr/lib/python3.9/runpy.py", line 87 in _run_code
  File "/usr/lib/python3.9/runpy.py", line 197 in _run_module_as_main
[2022-10-22 23:32:09] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:43550]
[2022-10-22 23:33:31] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:52140]

FFprobe output from your camera

ffprobe version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, rtsp, from '<camera_url>':
  Metadata:
    title           : Media Presentation
  Duration: N/A, start: 0.300000, bitrate: N/A
  Stream #0:0: Video: h264 (Baseline), yuvj420p(pc, bt709, progressive), 1440x2560, 20 fps, 20 tbr, 90k tbn, 40 tbc

Frigate stats

No response

Operating system

Other

Install method

Docker Compose

Coral version

USB

Network connection

Wired

Camera make and model

Hikvision/HiLook cameras + 1 Unifi G4 Pro

Any other information that may be helpful

No response

blakeblackshear commented 1 year ago

This is typically because the shm size is too small: https://docs.frigate.video/faqs#fatal-python-error-bus-error

psenakv commented 1 year ago

Thanks, it was probably caused by SHM size and it seems like it is working now.