Masonry with Lazysizes doesn't load images properly

[FullDome]

When using masonry with lazysizes, which lazyloads images, meaning it dynamically adds them to the grid, all added images that were not in the viewport do not fit in the grid as they should. They stacked. So I added a class to lazyload:

.lazyloading {
    opacity: 0;
    min-height: 350px;

But that doesn't solve the issue, since images have different heights, they overlap or have too much margin. When I resize the whole window even for just 1px, the images rearange themselves perfectly. I tried using imagesLoaded too, but it didn't help.

Test case: https://codepen.io/anon/pen/EpmpaN

[FullDome]

This temporarily solved the problem, by firing up masonry when scrolling:

var $grid = $('.grid').masonry({
  itemSelector: '.grid-item',
  percentPosition: true,
  columnWidth: '.grid-sizer'
});
// layout Masonry after each image loads
$grid.imagesLoaded().progress( function() {
    $grid.masonry('layout');
});  
// scroll event to fire masonry
    $(document).ready(function(){
    $(window).scroll(function(){
        if ($(window).scrollTop() > 300){
            $grid.masonry('layout');
        }
    });
});