search

kk7nc / RMDL

RMDL: Random Multimodel Deep Learning for Classification
https://rmdl.readthedocs.io/
GNU General Public License v3.0
424 stars 122 forks source link
classification cnn convolutional-neural-networks data-mining deep-learning deep-neural-networks dnn ensemble-learning image-classification information-retrieval keras machine-learning multimodel recurrent-neural-networks rnn tensorflow text-classification text-mining

readme

|DOI| |Best| |Pypi| |arxiv| |Download| |werckerstatus| |appveyor| |BuildStatus| |Join the chat at https://gitter.im/RMDL-Random-Multimodel-Deep-Learning| |PowerPoint| |researchgate| |Binder| |GitHublicense| |twitter|

Referenced paper : RMDL: Random Multimodel Deep Learning for Classification <https://arxiv.org/abs/1805.01890>__

Referenced paper : An Improvement of Data Classification Using Random Multimodel Deep Learning (RMDL) <https://arxiv.org/abs/1808.08121>__

Random Multimodel Deep Learning (RMDL):

A new ensemble, deep learning approach for classification. Deep learning models have achieved state-of-the-art results across many domains. RMDL solves the problem of finding the best deep learning structure and architecture while simultaneously improving robustness and accuracy through ensembles of deep learning architectures. RDML can accept as input a variety data to include text, video, images, and symbolic.

|RMDL|

Random Multimodel Deep Learning (RDML) architecture for classification. RMDL includes 3 Random models, oneDNN classifier at left, one Deep CNN classifier at middle, and one Deep RNN classifier at right (each unit could be LSTMor GRU).

Installation

There are pip and git for RMDL installation:

Using pip

.. code:: python

    pip install RMDL

Using git

.. code:: bash

git clone --recursive https://github.com/kk7nc/RMDL.git

The primary requirements for this package are Python 3 with Tensorflow. The requirements.txt file contains a listing of the required Python packages; to install all requirements, run the following:

.. code:: bash

pip -r install requirements.txt

Or

.. code:: bash

pip3  install -r requirements.txt

Or:

.. code:: bash

conda install --file requirements.txt

Documentation:

The exponential growth in the number of complex datasets every year requires more enhancement in machine learning methods to provide robust and accurate data classification. Lately, deep learning approaches have been achieved surpassing results in comparison to previous machine learning algorithms on tasks such as image classification, natural language processing, face recognition, and etc. The success of these deep learning algorithms relys on their capacity to model complex and non-linear relationships within data. However, finding the suitable structure for these models has been a challenge for researchers. This paper introduces Random Multimodel Deep Learning (RMDL): a new ensemble, deep learning approach for classification. RMDL solves the problem of finding the best deep learning structure and architecture while simultaneously improving robustness and accuracy through ensembles of deep learning architectures. In short, RMDL trains multiple models of Deep Neural Network (DNN), Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) in parallel and combines their results to produce better result of any of those models individually. To create these models, each deep learning model has been constructed in a random fashion regarding the number of layers and nodes in their neural network structure. The resulting RDML model can be used for various domains such as text, video, images, and symbolic. In this Project, we describe RMDL model in depth and show the results for image and text classification as well as face recognition. For image classification, we compared our model with some of the available baselines using MNIST and CIFAR-10 datasets. Similarly, we used four datasets namely, WOS, Reuters, IMDB, and 20newsgroup and compared our results with available baselines. Web of Science (WOS) has been collected by authors and consists of three sets~(small, medium and large set). Lastly, we used ORL dataset to compare the performance of our approach with other face recognition methods. These test results show that RDML model consistently outperform standard methods over a broad range of data types and classification problems.

Datasets for RMDL:

Text Datasets:

Image datasets:

Face Recognition

The Database of Faces (The Olivetti Faces Dataset) <http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html>__

Requirements for RMDL :

General:

GPU (if you want to run on GPU):

Text and Document Classification

Parameters:

Text_Classification

.. code:: python

     from RMDL import RMDL_Text

.. code:: python

   Text_Classification(x_train, y_train, x_test,  y_test, batch_size=128,
                    EMBEDDING_DIM=50,MAX_SEQUENCE_LENGTH = 500, MAX_NB_WORDS = 75000,
                    GloVe_dir="", GloVe_file = "glove.6B.50d.txt",
                    sparse_categorical=True, random_deep=[3, 3, 3], epochs=[500, 500, 500],  plot=True,
                    min_hidden_layer_dnn=1, max_hidden_layer_dnn=8, min_nodes_dnn=128, max_nodes_dnn=1024,
                    min_hidden_layer_rnn=1, max_hidden_layer_rnn=5, min_nodes_rnn=32,  max_nodes_rnn=128,
                    min_hidden_layer_cnn=3, max_hidden_layer_cnn=10, min_nodes_cnn=128, max_nodes_cnn=512,
                    random_state=42, random_optimizor=True, dropout=0.05):

Input


- x_train
- y_train
- x_test
- y_test

batch_size

EMBEDDING_DIM


- batch_size: Integer. Shape of word embedding (this number should be same with GloVe or other pre-trained embedding techniques that be used), it will default to 50 that used with pain of glove.6B.50d.txt file.

MAX_SEQUENCE_LENGTH

MAX_NB_WORDS


- MAX_NB_WORDS: Integer. Maximum number of unique words in datasets,  it will default to 75000.

GloVe_dir

GloVe_file


- GloVe_dir: String. Which version of GloVe or pre-trained word emending will be used,  it will default to glove.6B.50d.txt.

- NOTE: if you use other version of GloVe EMBEDDING_DIM must be same dimensions.

sparse_categorical

random_deep


- random_deep: Integer [3]. Number of ensembled model used in RMDL random_deep[0] is number of DNN, random_deep[1] is number of RNN, random_deep[0] is number of CNN, it will default to [3, 3, 3].

epochs

plot


- plot: bool. True: shows confusion matrix and accuracy and loss

min_hidden_layer_dnn

max_hidden_layer_dnn


- max_hidden_layer_dnn: Integer. Upper bounds of hidden layers of DNN used in RMDL, it will default to 8.

min_nodes_dnn

max_nodes_dnn


- max_nodes_dnn: Integer. Upper bounds of nodes in each layer of DNN used in RMDL, it will default to 1024.

min_hidden_layer_rnn

max_hidden_layer_rnn


- man_hidden_layer_rnn: Integer. Upper Bounds of hidden layers of RNN used in RMDL, it will default to 5.

min_nodes_rnn

max_nodes_rnn


- max_nodes_rnn: Integer. Upper bounds of nodes (LSTM or GRU) in each layer of RNN used in RMDL, it will default to 128.

min_hidden_layer_cnn

max_hidden_layer_cnn


- max_hidden_layer_cnn: Integer. Upper Bounds of hidden layers of CNN used in RMDL, it will default to 10.

min_nodes_cnn

max_nodes_cnn


- min_nodes_cnn: Integer. Upper bounds of nodes (2D convolution layer) in each layer of CNN used in RMDL, it will default to 512.

random_state

random_optimizor


- random_optimizor : bool, If False, all models use adam optimizer.  If True, all models use random optimizers. it will default to True

dropout

Image_Classification

.. code:: python

     from RMDL import RMDL_Image

.. code:: python

     Image_Classification(x_train, y_train, x_test, y_test, shape, batch_size=128,
                              sparse_categorical=True, random_deep=[3, 3, 3], epochs=[500, 500, 500], plot=True,
                              min_hidden_layer_dnn=1, max_hidden_layer_dnn=8, min_nodes_dnn=128, max_nodes_dnn=1024,
                              min_hidden_layer_rnn=1, max_hidden_layer_rnn=5, min_nodes_rnn=32, max_nodes_rnn=128,
                              min_hidden_layer_cnn=3, max_hidden_layer_cnn=10, min_nodes_cnn=128, max_nodes_cnn=512,
                              random_state=42, random_optimizor=True, dropout=0.05)

Input

- x_train
- y_train
- x_test
- y_test

shape

batch_size


- batch_size: Integer. Number of samples per gradient update. If unspecified, it will default to 128.

sparse_categorical

random_deep


- random_deep: Integer [3]. Number of ensembled model used in RMDL random_deep[0] is number of DNN, random_deep[1] is number of RNN, random_deep[0] is number of CNN, it will default to [3, 3, 3].

epochs

plot


- plot: bool. True: shows confusion matrix and accuracy and loss

min_hidden_layer_dnn

max_hidden_layer_dnn


- max_hidden_layer_dnn: Integer. Upper bounds of hidden layers of DNN used in RMDL, it will default to 8.

min_nodes_dnn

max_nodes_dnn


- max_nodes_dnn: Integer. Upper bounds of nodes in each layer of DNN used in RMDL, it will default to 1024.

min_nodes_rnn

max_nodes_rnn


- maz_nodes_rnn: Integer. Upper bounds of nodes (LSTM or GRU) in each layer of RNN used in RMDL, it will default to 128.

min_hidden_layer_cnn

max_hidden_layer_cnn


- max_hidden_layer_cnn: Integer. Upper Bounds of hidden layers of CNN used in RMDL, it will default to 10.

min_nodes_cnn

max_nodes_cnn


- min_nodes_cnn: Integer. Upper bounds of nodes (2D convolution layer) in each layer of CNN used in RMDL, it will default to 512.

random_state

random_optimizor


- random_optimizor : bool, If False, all models use adam optimizer.  If True, all models use random optimizers. it will default to True

dropout

Example

MNIST

Import Packages


.. code:: python

        from keras.datasets import mnist
        import numpy as np
        from RMDL import RMDL_Image as RMDL

Load Data

.. code:: python

    (X_train, y_train), (X_test, y_test) = mnist.load_data()
    X_train_D = X_train.reshape(X_train.shape[0], 28, 28, 1).astype('float32')
    X_test_D = X_test.reshape(X_test.shape[0], 28, 28, 1).astype('float32')
    X_train = X_train_D / 255.0
    X_test = X_test_D / 255.0
    number_of_classes = np.unique(y_train).shape[0]
    shape = (28, 28, 1)

Using RMDL


.. code:: python

        batch_size = 128
        sparse_categorical = 0
        n_epochs = [100, 100, 100]  ## DNN-RNN-CNN
        Random_Deep = [3, 3, 3]  ## DNN-RNN-CNN

        RMDL.Image_Classification(X_train, y_train, X_test, y_test,shape,
                             batch_size=batch_size,
                             sparse_categorical=True,
                             random_deep=Random_Deep,
                             epochs=n_epochs)
IMDB
-----

-  This dataset contains 50,000 documents with 2 categories.

Import Packages

.. code:: python

    import sys
    import os
    from RMDL import text_feature_extraction as txt
    from keras.datasets import imdb
    import numpy as np
    from RMDL import RMDL_Text as RMDL

Load Data


.. code:: python

        print("Load IMDB dataset....")
        (X_train, y_train), (X_test, y_test) = imdb.load_data(num_words=MAX_NB_WORDS)
        print(len(X_train))
        print(y_test)
        word_index = imdb.get_word_index()
        index_word = {v: k for k, v in word_index.items()}
        X_train = [txt.text_cleaner(' '.join(index_word.get(w) for w in x)) for x in X_train]
        X_test = [txt.text_cleaner(' '.join(index_word.get(w) for w in x)) for x in X_test]
        X_train = np.array(X_train)
        X_train = np.array(X_train).ravel()
        print(X_train.shape)
        X_test = np.array(X_test)
        X_test = np.array(X_test).ravel()

Using RMDL

.. code:: python

    batch_size = 100
    sparse_categorical = 0
    n_epochs = [100, 100, 100]  ## DNN--RNN-CNN
    Random_Deep = [3, 3, 3]  ## DNN--RNN-CNN

    RMDL.Text_Classification(X_train, y_train, X_test, y_test,
                         batch_size=batch_size,
                         sparse_categorical=sparse_categorical,
                         random_deep=Random_Deep,
                         epochs=n_epochs)

Web Of Science

Import Packages


.. code:: python

        from RMDL import text_feature_extraction as txt
        from sklearn.model_selection import train_test_split
        from RMDL.Download import Download_WOS as WOS
        import numpy as np
        from RMDL import RMDL_Text as RMDL

Load Data

.. code:: python

    path_WOS = WOS.download_and_extract()
    fname = os.path.join(path_WOS,"WebOfScience/WOS11967/X.txt")
    fnamek = os.path.join(path_WOS,"WebOfScience/WOS11967/Y.txt")
    with open(fname, encoding="utf-8") as f:
        content = f.readlines()
        content = [txt.text_cleaner(x) for x in content]
    with open(fnamek) as fk:
        contentk = fk.readlines()
    contentk = [x.strip() for x in contentk]
    Label = np.matrix(contentk, dtype=int)
    Label = np.transpose(Label)
    np.random.seed(7)
    print(Label.shape)
    X_train, X_test, y_train, y_test = train_test_split(content, Label, test_size=0.2, random_state=4)

Using RMDL

.. code:: python

        batch_size = 100
        sparse_categorical = 0
        n_epochs = [5000, 500, 500]  ## DNN--RNN-CNN
        Random_Deep = [3, 3, 3]  ## DNN--RNN-CNN

        RMDL.Text_Classification(X_train, y_train, X_test, y_test,
                             batch_size=batch_size,
                             sparse_categorical=True,
                             random_deep=Random_Deep,
                             epochs=n_epochs,no_of_classes=12)

Reuters-21578
-------------

- This dataset contains 21,578 documents with 90 categories.

Import Packages

.. code:: python

     import sys
     import os
     import nltk
     nltk.download("reuters")
     from nltk.corpus import reuters
     from sklearn.preprocessing import MultiLabelBinarizer
     import numpy as np
     from RMDL import RMDL_Text as RMDL

Load Data

.. code:: python

         documents = reuters.fileids()

         train_docs_id = list(filter(lambda doc: doc.startswith("train"),
                                   documents))
         test_docs_id = list(filter(lambda doc: doc.startswith("test"),
                                  documents))
         X_train = [(reuters.raw(doc_id)) for doc_id in train_docs_id]
         X_test = [(reuters.raw(doc_id)) for doc_id in test_docs_id]
         mlb = MultiLabelBinarizer()
         y_train = mlb.fit_transform([reuters.categories(doc_id)
                                    for doc_id in train_docs_id])
         y_test = mlb.transform([reuters.categories(doc_id)
                               for doc_id in test_docs_id])
         y_train = np.argmax(y_train, axis=1)
         y_test = np.argmax(y_test, axis=1)

Using RMDL

.. code:: python

    batch_size = 100
    sparse_categorical = 0
    n_epochs = [20, 500, 50]  ## DNN--RNN-CNN
    Random_Deep = [3, 0, 0]  ## DNN--RNN-CNN

    RMDL.Text_Classification(X_train, y_train, X_test, y_test,
                 batch_size=batch_size,
                 sparse_categorical=True,
                 random_deep=Random_Deep,
                 epochs=n_epochs)

Olivetti Faces

Import Packages


.. code:: python

         from sklearn.datasets import fetch_olivetti_faces
         from sklearn.model_selection import train_test_split
         from RMDL import RMDL_Image as RMDL

Load Data

.. code:: python

     number_of_classes = 40
     shape = (64, 64, 1)
     data = fetch_olivetti_faces()
     X_train, X_test, y_train, y_test = train_test_split(data.data,
                                                   data.target, stratify=data.target, test_size=40)
     X_train = X_train.reshape(X_train.shape[0], 64, 64, 1).astype('float32')
     X_test = X_test.reshape(X_test.shape[0], 64, 64, 1).astype('float32')

Using RMDL


.. code:: python

        batch_size = 100
        sparse_categorical = 0
        n_epochs = [500, 500, 50]  ## DNN--RNN-CNN
        Random_Deep = [0, 0, 1]  ## DNN--RNN-CNN

        RMDL.Image_Classification(X_train, y_train, X_test, y_test,
                              shape,
                              random_optimizor=False,
                              batch_size=batch_size,
                              random_deep=Random_Deep,
                              epochs=n_epochs)

More Example
`link <https://github.com/kk7nc/RMDL/tree/master/Examples>`__

|Results|

Error and Comments:
----------------------

Send an email to kk7nc@virginia.edu

Citations
---------

.. code::

    @inproceedings{Kowsari2018RMDL,
         author = {Kowsari, Kamran and Heidarysafa, Mojtaba and Brown, Donald E. and Meimandi, Kiana Jafari and Barnes, Laura E.},
         title = {RMDL: Random Multimodel Deep Learning for Classification},
         booktitle = {Proceedings of the 2Nd International Conference on Information System and Data Mining},
         series = {ICISDM '18},
         year = {2018},
         isbn = {978-1-4503-6354-9},
         location = {Lakeland, FL, USA},
         pages = {19--28},
         numpages = {10},
         url = {http://doi.acm.org/10.1145/3206098.3206111},
         doi = {10.1145/3206098.3206111},
         acmid = {3206111},
         publisher = {ACM},
         address = {New York, NY, USA},
         keywords = {Data Mining, Deep Learning, Deep Neural Networks, Image Classification, Supervised Learning, Text Classification},
        } 

and 

.. code::

    @article{Heidarysafa2018RMDL,
    title={An Improvement of Data Classification Using Random Multimodel Deep Learning (RMDL)},
    author={Heidarysafa, Mojtaba and Kowsari, Kamran and Brown, Donald E. and Jafari Meimandi, Kiana and Barnes, Laura E.},
    booktitle={International Journal of Machine Learning and Computing (IJMLC)},
    year={2018},
    Volume={8},
    Number={4},
    pages={298--310},
    DOI={https://doi.org/10.18178/ijmlc.2018.8.4.703}
    }

.. |werckerstatus| image:: https://app.wercker.com/status/3a564158e809971e2f7416beba5d05af/s/master
   :target: https://app.wercker.com/project/byKey/3a564158e809971e2f7416beba5d05af
.. |BuildStatus| image:: https://travis-ci.org/kk7nc/RMDL.svg?branch=master
   :target: https://travis-ci.org/kk7nc/RMDL
.. |PowerPoint| image:: https://img.shields.io/badge/Presentation-download-red.svg?style=flat
   :target: https://github.com/kk7nc/RMDL/blob/master/docs/RMDL.pdf
.. |researchgate| image:: https://img.shields.io/badge/ResearchGate-RMDL-blue.svg?style=flat
   :target: https://www.researchgate.net/publication/325936017_RMDL_Random_Multimodel_Deep_Learning_for_Classification
.. |Binder| image:: https://mybinder.org/badge.svg
   :target: https://mybinder.org/v2/gh/kk7nc/RMDL/master
.. |pdf| image:: https://img.shields.io/badge/pdf-download-red.svg?style=flat
   :target: https://github.com/kk7nc/RMDL/blob/master/docs/ACM-RMDL.pdf
.. |GitHublicense| image:: https://img.shields.io/badge/licence-GPL-blue.svg
   :target: ./LICENSE
.. |RDL| image:: docs/RDL.jpg
.. |RMDL| image:: docs/RMDL.jpg
.. |Results| image:: docs/RMDL_Results.png
.. |Data| image:: https://img.shields.io/badge/DOI-10.17632/9rw3vkcfy4.6-blue.svg?style=flat
   :target: http://dx.doi.org/10.17632/9rw3vkcfy4.6
.. |Pypi| image:: https://img.shields.io/badge/Pypi-RMDL%201.0.8-blue.svg
   :target: https://pypi.org/project/RMDL/
.. |DOI| image:: https://img.shields.io/badge/DOI-10.1145/3206098.3206111-blue.svg?style=flat
   :target: https://doi.org/10.1145/3206098.3206111
.. |appveyor| image:: https://ci.appveyor.com/api/projects/status/github/kk7nc/RMDL?branch=master&svg=true
    :target: https://ci.appveyor.com/project/kk7nc/RMDL
.. |arxiv| image:: https://img.shields.io/badge/arXiv-1805.01890-red.svg
    :target: https://arxiv.org/abs/1805.01890
.. |twitter| image:: https://img.shields.io/twitter/url/http/shields.io.svg?style=social
    :target: https://twitter.com/intent/tweet?text=RMDL:%20Random%20Multimodel%20Deep%20Learning%20for%20Classification%0aGitHub:&url=https://github.com/kk7nc/RMDL&hashtags=DeepLearning,classification,MachineLearning,deep_neural_networks,Image_Classification,Text_Classification,EnsembleLearning
.. |Join the chat at https://gitter.im/RMDL-Random-Multimodel-Deep-Learning| image:: https://badges.gitter.im/Join%20Chat.svg
   :target: https://gitter.im/RMDL-Random-Multimodel-Deep-Learning/Lobby?source=orgpage

.. |Download| image:: https://static.pepy.tech/personalized-badge/rmdl?period=total&units=international_system&left_color=grey&right_color=brightgreen&left_text=Downloads
 :target: https://pepy.tech/project/rmdl

.. |Best| image::     https://img.shields.io/badge/Award-Best%20Paper%20Award-brightgreen
    :target: http://www.icisdm.org/icisdm2018.html