How to load and data (non-image) for logistic regression ?

[salemmohammed]

Hi all,

I am trying to write a logistic regression code for distributed machine. I have my data label and vector of features example label -> features 0 , 00.1 , 0.1, .... 1 , 0.1 , 0.002 ...

My general structure as the following:

Define the network symbol, equivalent to logistic regression

net = mx.symbol.Variable('data') net = mx.symbol.FullyConnected(data=net, name='fc1', num_hidden=1) net = mx.symbol.LinearRegressionOutput(data=net, name='logistic regression')

Load data ????? I do not know how to do loading data for my data .csv

How to do iteration ?

model = mx.model.FeedForward.create(symbol=net, X=?, num_epoch=20, learning_rate=0.1)

Make prediction

model.predict(?)

Can anybody who familiar with MXNet give me some guidance ?

Sincerely

[qinhui99]

You can see these codes:

from sklearn.datasets import fetch_california_housing from sklearn.preprocessing import StandardScaler import numpy as np import mxnet as mx from sklearn.utils import shuffle import logging

We get the california house datasets

housing = fetch_california_housing() m, n = housing.data.shape

print (m,n)

housing_data_plusbias = np.c[np.ones((m, 1)), housing.data]

scaler = StandardScaler() scaled_housing_data = scaler.fit_transform(housing.data) scaled_housing_data_plusbias = np.c[np.ones((m, 1)), scaled_housing_data]

n_epochs = 15 learning_rate = 0.025

batch_size =128

shuffle data

X, y = shuffle(scaled_housing_data_plus_bias, housing.target)

X, y = (scaled_housing_data_plus_bias, housing.target) print (len(X),X[0].shape,y[0])

Define the MLP model

x_sym = mx.symbol.Variable('data') y_sym = mx.symbol.Variable('softmax_label')

fc1 = mx.symbol.FullyConnected(data=x_sym, num_hidden=40, name='pre') act1 = mx.symbol.Activation(data = fc1, name='act1', act_type="relu") fc2 = mx.sym.FullyConnected(data=act1, name='fc2', num_hidden=20) act2 = mx.symbol.Activation(data = fc2, name='act2', act_type="relu") fc3 = mx.sym.FullyConnected(data=act2, name='fc3', num_hidden=1) loss = mx.symbol.LinearRegressionOutput(data=fc3,label=y_sym, name='loss')

Binding the model

model = mx.model.FeedForward( ctx=mx.cpu(), symbol=loss, num_epoch=n_epochs, learning_rate=learning_rate, optimizer='adam' ) logging.basicConfig(level=logging.INFO)

Build iterator

slice_index=20500 train_iter = mx.io.NDArrayIter(data=X[:18000], label=y[:18000], batch_size=batch_size, shuffle=True) eval_iter = mx.io.NDArrayIter(data=X[18000:slice_index], label=y[18000:slice_index], batch_size=batch_size, shuffle=True)

test_iter=mx.io.NDArrayIter(data=X[slice_index:slice_index+1], shuffle=False)

We want the mse and rmse

eval_metrics = ['mse'] eval_metrics.append('rmse')

Begin to train

model.fit(X = train_iter, eval_metric=eval_metrics, eval_data=eval_iter)

Predit

r=model.predict(test_iter)

check the prediction

print (r,y[slice_index:slice_index+1])

Hope it can help you.

[salemmohammed]

@qinhui99

I tried your example but I could not because the data iteration and loading.

Here is my code. I am having a lot of nan and data kept the same without change.

import os import mxnet as mx import numpy as np import logging

batch = 500 shape = (9999,)

Define the network symbol, equivalent to logistic regression

def get_net(): data = mx.symbol.Variable('data') fc1 = mx.symbol.FullyConnected(data=data, name='fc1', num_hidden=1) net = mx.symbol.LogisticRegressionOutput(data=fc1, name='softmax') return net

KVStore

kv = mx.kvstore.create('dist_sync')

data iterator

network = get_net() devs = [mx.cpu(0)]

data_train = mx.io.CSVIter( data_csv='./data.csv', data_shape=shape, label_csv='./label.csv', label_shape=(1,), batch_size=batch)

logging.basicConfig(level=logging.DEBUG)

optimizer_params = {'learning_rate': 0.1}

monitor = mx.mon.Monitor(1, pattern=".*") if 1 > 0 else None

eval_metrics = ['accuracy']

callbacks that run after each batch

batch_end_callbacks = [mx.callback.Speedometer(batch, 10)]

training

mod = mx.mod.Module(symbol=network, context=devs, data_names=['data'], label_names=['softmax_label'])

mod.fit(data_train, num_epoch = 100, eval_metric = eval_metrics, kvstore = kv, optimizer = 'sgd', optimizer_params = optimizer_params, batch_end_callback = batch_end_callbacks, allow_missing = True, monitor = monitor)

Can anybody notice what is the problem with my code or data.

data.csv is each line has 9999 data such as 0.1,0.001,...... label.csv. has total 10000 lines and each line has 0 or 1.

Here is the output:

INFO:root:Epoch[0] Batch [10] Speed: 4023.28 samples/sec Train-accuracy=0.501636 INFO:root:Epoch[0] Batch [10] Speed: 3880.19 samples/sec Train-accuracy=0.501636 INFO:root:Epoch[0] Batch [10] Speed: 3908.93 samples/sec Train-accuracy=0.501636 INFO:root:Epoch[0] Train-accuracy=0.492222 INFO:root:Epoch[0] Time cost=2.485 INFO:root:Epoch[0] Train-accuracy=0.492222 INFO:root:Epoch[0] Time cost=2.524 INFO:root:Epoch[0] Train-accuracy=0.492222 INFO:root:Epoch[0] Time cost=2.550

[qinhui99]

Your problem is a classifier problem. So you should change the get_net() function like these:

net = mx.sym.Variable('data') net = mx.sym.FullyConnected(net, name='fc1', num_hidden=20) net = mx.sym.Activation(net, name='relu1', act_type="relu")

Because you have two labels(0 or 1), so the num_hidden should be 2.

net = mx.sym.FullyConnected(net, name='fc2', num_hidden=2)

For classifier problem ,you can not use LogisticRegressionOutput as the output.

You should use SoftmaxOutput.

net = mx.sym.SoftmaxOutput(net, name='softmax')

[salemmohammed]

@qinhui99 The problem is still the same. I modified the logistic regression to softmax with two hidden layer.

INFO:root:Epoch[0] Batch [10] Speed: 7542.74 samples/sec Train-accuracy=0.501636 INFO:root:Epoch[0] Batch [10] Speed: 7466.22 samples/sec Train-accuracy=0.501636 INFO:root:Epoch[0] Batch [10] Speed: 7357.59 samples/sec Train-accuracy=0.501636 INFO:root:Epoch[0] Train-accuracy=0.492222 INFO:root:Epoch[0] Time cost=1.505 INFO:root:Epoch[0] Train-accuracy=0.492222 INFO:root:Epoch[0] Time cost=1.514 INFO:root:Epoch[0] Train-accuracy=0.492222 INFO:root:Epoch[0] Time cost=1.521 INFO:root:Epoch[1] Batch [10] Speed: 6425.68 samples/sec Train-accuracy=0.501636 INFO:root:Epoch[1] Batch [10] Speed: 6431.58 samples/sec Train-accuracy=0.501636 INFO:root:Epoch[1] Batch [10] Speed: 6353.69 samples/sec Train-accuracy=0.501636

[salemmohammed]

@piiswrong I am trying to use binary classification logistic regression with MXNet. I am working on 3 workers and one ps. I have the code as following:

batch = 500 shape = (9999,) data = mx.symbol.Variable('data') fc1 = mx.symbol.FullyConnected(data=data, name='fc1', num_hidden=1) net = mx.symbol.LogisticRegressionOutput(data=fc1, name='softmax')

kv = mx.kvstore.create('dist_sync')

network = get_net() devs = [mx.cpu(0)] data_train = mx.io.CSVIter( data_csv='./data.csv', data_shape=shape, label_csv='./label.csv', label_shape=(1,), batch_size=batch) logging.basicConfig(level=logging.DEBUG) optimizer_params = {'learning_rate': 0.1} monitor = mx.mon.Monitor(1, pattern=".*") if 1 > 0 else None eval_metrics = ['accuracy']

callbacks that run after each batch batch_end_callbacks = [mx.callback.Speedometer(batch, 10)]

mod = mx.mod.Module(symbol=network,context=devs,data_names=['data'], label_names=['softmax_label'])

mod.fit(data_train,num_epoch = 100, eval_metric = eval_metrics,kvstore = kv,optimizer = 'sgd', optimizer_params = optimizer_params,batch_end_callback = batch_end_callbacks, allow_missing = True,monitor = monitor)

I have two labels 0 and 1 and I have 9999 data per line example label.csv data.csv 0 [0.1,0.5................>9999 ]

The training accuracy is the same.

INFO:root:Epoch[0] Batch [10] Speed: 7542.74 samples/sec Train-accuracy=0.501636 INFO:root:Epoch[0] Batch [10] Speed: 7466.22 samples/sec Train-accuracy=0.501636 INFO:root:Epoch[0] Batch [10] Speed: 7357.59 samples/sec Train-accuracy=0.501636 INFO:root:Epoch[0] Train-accuracy=0.492222 INFO:root:Epoch[0] Time cost=1.505 INFO:root:Epoch[0] Train-accuracy=0.492222 INFO:root:Epoch[0] Time cost=1.514 INFO:root:Epoch[0] Train-accuracy=0.492222 INFO:root:Epoch[0] Time cost=1.521 INFO:root:Epoch[1] Batch [10] Speed: 6425.68 samples/sec Train-accuracy=0.501636 INFO:root:Epoch[1] Batch [10] Speed: 6431.58 samples/sec Train-accuracy=0.501636 INFO:root:Epoch[1] Batch [10] Speed: 6353.69 samples/sec Train-accuracy=0.501636

Can anybody point me where is my problem ?

[qinhui99]

Why don't you use the MLP model instead of the only one layer ? That means: data = mx.symbol.Variable('data') net = mx.sym.FullyConnected(data, name='fc1', num_hidden=12) net = mx.sym.Activation(net, name='relu1', act_type="relu") net = mx.symbol.FullyConnected(net, name='fc2', num_hidden=2) net = mx.symbol.SoftmaxOutput(net, name='softmax')

I tested three different classifier datasets, I got the same problem with you when I use the only one layer. (I guess that mxnet codes are not suitable to do such simple jobs.) When I used the MLP model, the results were better. So I suggest you to use MLP.

Good luck!

[szha]

This issue is closed due to lack of activity in the last 90 days. Feel free to ping me to reopen if this is still an active issue. Thanks!