Sketched-Webpages-Generator

Customizable open-source software to generate randomized sketched web-pages. Article on Medium

Introduction

Mapping hand sketches to computer code is a modern AI problem that has been under research for some time. One of the main example applications for this problem is the generation of web-page front-end code from a hand sketch. Training a deep learning model for this task requires a large dataset that contains a wide variety of samples, but sketching a large number of web-pages is a tedious and time-consuming task. The software we publish is an efficient tool to generate any number of unique sketched web-pages.

The Idea

This program was one main part of our graduation project, in which we developed an end-to-end deep learning model to output web-page code given an input sketch. We could not find any suitable dataset for our case, so we built a dataset generator that creates randomized web-pages then turns them into matching sketches.

How It Works

We have created a DSL (domain-specific language) dictionary in which we map a block of code to a single token to simplify our problem. The generator creates unique random pages using the DSL dictionary and applying a set of rules to output DSL files which are then mapped to realistic looking web-pages using a DSL compiler. The resulting web-pages are rendered with a special CSS file using PhantomJS engine. Finally we apply simple object detection on the rendered web-pages to detect all the different elements within this page, and create a matching sketch for each web-page; the sketch is generated by placing an actual hand sketch for the detected element, the sketched element is chosen randomly from a set of images provided for this element.

Here is the block diagram for the generator:

The main rule is to follow a correct web-page structure when creating a new randomized page.

Mapping the DSL to HTML code.

The rendered web-page without and with the special CSS file applied:

The element detection process and sketched output:

Features

• Generate any number of unique web-pages.
• Generate any number of different sketches for each web-page.
• Automatically save batches of generated sketches in zip files.
• Ability to stop and resume the generation process keeping the output uniqueness.
• Option to save intermediate outputs.

Prerequisites

• Python 3.6
• Dependencies: pip install cv2 numpy selenium imutils imagesize

Usage

python main.py --help

Main.py [-h] --number NUMBER [--fresh] [--variations VARIATIONS]
               [--intermediate] [--height HEIGHT] [--zipping]
               [--batchsize BATCHSIZE] [--verbose]

optional arguments:
  -h, --help            show this help message and exit
  --number NUMBER, -n NUMBER
                        Number of samples to be generated. Starts with n=0 if
                        the output directory is empty.
  --fresh, -f           Fresh start; removes any existing outputs.
  --variations VARIATIONS, -v VARIATIONS
                        Number of different sketches for each generated
                        webpage.
  --intermediate, -i    Save intermediate outputs from rendering during
                        generation process.
  --height HEIGHT       Specifiy page height in pixels. Note: page width is
                        1200px
  --zipping, -z         Store batches of output files as zipped files. Default
                        batch size is 500.
  --batchsize BATCHSIZE, -s BATCHSIZE
                        Number of pages to be zipped together.
  --verbose             Printing in console during execution.

Known Issues

• Currently, data augmentation is done by generating a variety of sketches for each web-page. Creating different modified versions of the same sketch is disabled as it needs more tuning.
• Elements may intersect or overlap. This highly depends on the provided sketched elements.
• Sketched element random selection needs more biasing to choose close size and aspect ratio elements, and therefore we need more variations of sketched elements for each element.
• Some performance optimizations need to be done.

Citations

Pix2code (Tony Beltramelli)

• The DSL method was mainly inspired by pix2code project by Toni.
• The DSL compiler we use is a developed version of pix2code DSL compiler.

Startbootstrap

• We used simple startbootstrap templates as a reference for our generated web-pages.

PhantomJS

• We use phantomjs engine to render the web-pages using python.

License

MIT License Copyright (c) 2019 - Team Memebers Abdelrahman Abdelhamid, Ahmed Bally, Eman El-Sheikh, Abdelrahman Metwally, Salma Elkhawaga and Mostafa Tarek.