bradtraversy / proshop_mern

Shopping cart built with MERN & Redux
1.99k stars 1.18k forks source link

Images are uploaded to the server would eventually get deleted. #90

Closed Slakie closed 3 years ago

Slakie commented 3 years ago

Issue: Images are uploaded to the server would eventually get disappeared or deleted after a while.

Resolution: -> https://help.heroku.com/K1PPS2WM/why-are-my-file-uploads-missing-deleted The Heroku filesystem is ephemeral - that means that any changes to the filesystem whilst the dyno is running only last until that dyno is shut down or restarted. Each dyno boots with a clean copy of the filesystem from the most recent deploy. This is similar to how many container based systems, such as Docker, operate.

In addition, under normal operations dynos will restart every day in a process known as "Cycling".

These two facts mean that the filesystem on Heroku is not suitable for persistent storage of data. In cases where you need to store data we recommend using a database addon such as Postgres (for data) or a dedicated file storage service such as AWS S3 (for static files). If you don't want to set up an account with AWS to create an S3 bucket we also have addons here that handle storage and processing of static assets https://elements.heroku.com/addons

talmax1124 commented 3 years ago

@Slakie Did you implement this already the S3? Can you help me out?

Slakie commented 3 years ago

@talmax1124 I did something else instead of AWS S3 buckets, there is a good image/video service called Cloudinary they have a free plan which is very convenient for a project like this in my opinion... If you wish I can share the code?

eliashezron commented 3 years ago

On Sun, 10 Jan 2021 at 16:15, Slakie notifications@github.com wrote:

@talmax1124 https://github.com/talmax1124 I did something else instead of AWS S3 buckets, there is a good image/video service called Cloudinary they have a free plan which is very convenient for a project like this in my opinion... If you wish I can share the code?

@slakie please do.

β€” You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bradtraversy/proshop_mern/issues/90#issuecomment-757474468, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOQNFUSTUZZSGWXB5JYQR7TSZGR53ANCNFSM4V3KKMJA .

-- Opio Elias Hezron Makerere University, Faculty of Technology.

Slakie commented 3 years ago

@talmax1124 @eliashezron

  1. install 2 additional dependencies for the backend: "cloudinary" and "multer-storage-cloudinary"
  2. Sign up at cloudinary.com
  3. Copy your auto generated Environment variables from: cloudinary.com -> Dashboard
  4. Change your existing file code: backend -> routes -> uploadRoutes.js

onto:

import path from 'path'; import express from 'express'; import dotenv from 'dotenv'; import cloudinary from 'cloudinary'; import { CloudinaryStorage } from 'multer-storage-cloudinary'; import multer from 'multer';

dotenv.config();

const cloud = cloudinary.v2; const router = express.Router();

cloud.config({ cloud_name: process.env.CLOUDINARY_CLOUD_NAME, // Environment variables from your cloudinary account api_key: process.env.CLOUDINARY_API_KEY, api_secret: process.env.CLOUDINARY_API_SECRET, });

const storage = new CloudinaryStorage({ cloudinary: cloud, params: { folder: 'FOLDER_NAME', // any desirable name public_id: (req, file) => ${file.originalname.split('.')[0]}-${Date.now()}, }, });

function checkFileType(file, cb) { const filetypes = /jpg|jpeg|png/; const extname = filetypes.test( path.extname(file.originalname).toLocaleLowerCase() ); const mimetype = filetypes.test(file.mimetype); if (extname && mimetype) { return cb(null, true); } else { cb(null, false); } }

const upload = multer({ storage, fileFilter: function (req, file, cb) { checkFileType(file, cb); }, });

router.post('/', upload.single('image'), (req, res) => { res.send(req.file.path); });

export default router;

eliashezron commented 3 years ago

Awesome, thanks! @slakie

On Sun, 10 Jan 2021 at 17:26, Slakie notifications@github.com wrote:

@talmax1124 https://github.com/talmax1124 @eliashezron https://github.com/eliashezron

  1. install 2 additional dependencies for the backend: "cloudinary" and "multer-storage-cloudinary"

  2. Sign up at cloudinary.com

  3. Copy your auto generated Environment variables from: cloudinary.com -> Dashboard

  4. Change your existing file code: backend -> routes -> uploadRoutes.js onto:

    `import path from 'path';

    import express from 'express';

    import dotenv from 'dotenv';

    import cloudinary from 'cloudinary';

    import { CloudinaryStorage } from 'multer-storage-cloudinary';

    import multer from 'multer';

dotenv.config();

const cloud = cloudinary.v2;

const router = express.Router();

cloud.config({

cloud_name: process.env.CLOUDINARY_CLOUD_NAME,

api_key: process.env.CLOUDINARY_API_KEY,

api_secret: process.env.CLOUDINARY_API_SECRET,

});

const storage = new CloudinaryStorage({

cloudinary: cloud,

params: {

folder: 'slakieshop',

public_id: (req, file) =>

${file.originalname.split('.')[0]}-${Date.now()},

},

});

function checkFileType(file, cb) {

const filetypes = /jpg|jpeg|png/;

const extname = filetypes.test(

path.extname(file.originalname).toLocaleLowerCase()

);

const mimetype = filetypes.test(file.mimetype);

if (extname && mimetype) {

return cb(null, true);

} else {

cb(null, false);

}

}

const upload = multer({

storage,

fileFilter: function (req, file, cb) {

checkFileType(file, cb);

},

});

router.post('/', upload.single('image'), (req, res) => {

res.send(req.file.path);

});

export default router;`

β€” You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/bradtraversy/proshop_mern/issues/90#issuecomment-757484983, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOQNFUUL2TF32QUDZ5K3CULSZG2K3ANCNFSM4V3KKMJA .

-- Opio Elias Hezron Makerere University, Faculty of Technology.

Slakie commented 3 years ago

@eliashezron you are welcome! Hopefully it is clear enough πŸ˜ƒ there will be missing backticks... could not add them in here lol

eliashezron commented 3 years ago

@slakie I’ll manage πŸ˜‰

On Sun, 10 Jan 2021 at 17:42, Slakie notifications@github.com wrote:

@eliashezron https://github.com/eliashezron you are welcome! Hopefully it is clear enough πŸ˜ƒ there will be missing backticks... could not add them in here lol

β€” You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/bradtraversy/proshop_mern/issues/90#issuecomment-757487549, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOQNFUULLNEZB5IRXTBNKWTSZG4ELANCNFSM4V3KKMJA .

-- Opio Elias Hezron Makerere University, Faculty of Technology.

talmax1124 commented 3 years ago

@Slakie Your code for the

const storage = new CloudinaryStorage({

cloudinary: cloud,

params: {

folder: 'slakieshop',

public_id: (req, file) => ${file.originalname.split('.')[0]}-${Date.now()}, }, }); Is written wrong, can you provide the right one?
talmax1124 commented 3 years ago

Can you share your repo link @Slakie ?

talmax1124 commented 3 years ago

& is there a way to make images 640 x 510? Read some documentation, don't understand it. @slake

talmax1124 commented 3 years ago

@eliashezron @Slakie

Slakie commented 3 years ago

@talmax1124 Hello Carlos, below is the correction... (there were backticks missing)

const storage = new CloudinaryStorage({
    cloudinary: cloud,
    params: {
        folder: 'folder_name', // any desirable name
        public_id: (req, file) => `${file.originalname.split('.')[0]}-${Date.now()}`,
    },
});
talmax1124 commented 3 years ago

& is the only file I need to redo or do we need to edit the product screen?

On Sun, Jan 10, 2021 at 6:20 PM Slakie notifications@github.com wrote:

@eliashezron https://github.com/eliashezron: Below is the correction... (there were backticks missing)

const storage = new CloudinaryStorage({ cloudinary: cloud, params: { folder: 'folder_name', // any desirable name public_id: (req, file) => ${file.originalname.split('.')[0]}-${Date.now()}, }, });

β€” You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/bradtraversy/proshop_mern/issues/90#issuecomment-757562052, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOVNS3UNKJZTUT3SKW6F6ITSZIY3NANCNFSM4V3KKMJA .

Slakie commented 3 years ago

& is the only file I need to redo or do we need to edit the product screen? … On Sun, Jan 10, 2021 at 6:20 PM Slakie @.***> wrote: @eliashezron https://github.com/eliashezron: Below is the correction... (there were backticks missing) const storage = new CloudinaryStorage({ cloudinary: cloud, params: { folder: 'folder_name', // any desirable name public_id: (req, file) => ${file.originalname.split('.')[0]}-${Date.now()}, }, }); β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#90 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOVNS3UNKJZTUT3SKW6F6ITSZIY3NANCNFSM4V3KKMJA .

yeah that's the only file, let me know if it is all working for you...

talmax1124 commented 3 years ago

I am confused on the folder part. Do we need to make a folder or does it do it by iteslf. & how do I make it a certain size always? @Slakie

talmax1124 commented 3 years ago

@Slakie ,

It says this:

file:///Users/carlosdiazplaza/Documents/GitHub/shop/backend/routes/uploadRoutes.js:44 [0] return cb(null, true); [0] ^^^^^^ [0] [0] SyntaxError: Illegal return statement [0] at Loader.moduleStrategy (internal/modules/esm/translators.js:141:18) [0] [nodemon] app crashed - waiting for file changes before starting... [1] /Users/carlosdiazplaza/Documents/GitHub/shop/frontend/node_modules/dir-glob/index.js:72 [1] const globs = [].concat(input).map(x => pathType.isDirectorySync(getPath(x, options.cwd)) ? getGlob(x, options) : x); [1] ^ [1] [1] TypeError: pathType.isDirectorySync is not a function [1] at /Users/carlosdiazplaza/Documents/GitHub/shop/frontend/node_modules/dir-glob/index.js:72:51 [1] at Array.map () [1] at module.exports.sync (/Users/carlosdiazplaza/Documents/GitHub/shop/frontend/node_modules/dir-glob/index.js:72:33) [1] at globDirs (/Users/carlosdiazplaza/Documents/GitHub/shop/frontend/node_modules/globby/index.js:91:9) [1] at getPattern (/Users/carlosdiazplaza/Documents/GitHub/shop/frontend/node_modules/globby/index.js:94:67) [1] at /Users/carlosdiazplaza/Documents/GitHub/shop/frontend/node_modules/globby/index.js:142:19 [1] at Array.reduce () [1] at module.exports.sync (/Users/carlosdiazplaza/Documents/GitHub/shop/frontend/node_modules/globby/index.js:141:26) [1] at verifyNoTypeScript (/Users/carlosdiazplaza/Documents/GitHub/shop/frontend/node_modules/react-scripts/scripts/utils/verifyTypeScriptSetup.js:42:27) [1] at verifyTypeScriptSetup (/Users/carlosdiazplaza/Documents/GitHub/shop/frontend/node_modules/react-scripts/scripts/utils/verifyTypeScriptSetup.js:64:9) [1] npm ERR! code ELIFECYCLE [1] npm ERR! errno 1 [1] npm ERR! frontend@0.1.0 start: react-scripts start npm ERR! Exit status 1 [1] npm ERR! [1] npm ERR! Failed at the frontend@0.1.0 start script. [1] npm ERR! This is probably not a problem with npm. There is likely additional logging output above. [1] [1] npm ERR! A complete log of this run can be found in: [1] npm ERR! /Users/carlosdiazplaza/.npm/_logs/2021-01-11T00_56_26_219Z-debug.log [1] npm ERR! code ELIFECYCLE [1] npm ERR! errno 1 [1] npm ERR! creative-duo-shop@1.0.0 client: npm start --prefix frontend npm ERR! Exit status 1 [1] npm ERR! [1] npm ERR! Failed at the creative-duo-shop@1.0.0 client script. [1] npm ERR! This is probably not a problem with npm. There is likely additional logging output above. [1] [1] npm ERR! A complete log of this run can be found in: [1] npm ERR! /Users/carlosdiazplaza/.npm/_logs/2021-01-11T00_56_26_258Z-debug.log [1] npm run client exited with code 1

Slakie commented 3 years ago

& is there a way to make images 640 x 510? Read some documentation, don't understand it. @slake

There are many ways in images manipulation, which's why I picked cloudinary... It is very easy to setup your preferable way of images transformation: One of the ways I like and found it very convenient for me is:

then do this in your code:

const storage = new CloudinaryStorage({
    cloudinary: cloud,
    params: {
        folder: 'folder_name', // any desirable name
        transformation: '640x510center', // the name you saved in transformation presets
        public_id: (req, file) => `${file.originalname.split('.')[0]}-${Date.now()}`,
    },
});
talmax1124 commented 3 years ago

& is there a way to make images 640 x 510? Read some documentation, don't understand it. @slake

There are many ways in images manipulation, which's why I picked cloudinary... It is very easy to setup your preferable way of images transformation: One of the ways I like and found it very convenient for me is:

  • go to your cloudinary account, there is "Transformations" tab, add you desirable preset for example, you need width 640px and height 510px then change the Mode let's say "fill", in Gravity choose "center"... once you done with your settings then Save As give it a name let's say: 640x510center

then do this in your code:

const storage = new CloudinaryStorage({ cloudinary: cloud, params: { folder: 'folder_name', // any desirable name transformation: '640x510center', // the name you saved in transformation presets public_id: (req, file) => ${file.originalname.split('.')[0]}-${Date.now()}, }, });

How can I fix the error in the post above? @Slakie

talmax1124 commented 3 years ago

@Slakie Do you have any fixes?

Slakie commented 3 years ago

@talmax1124 looks like you have a typo somewhere, show me your code in uploadRoutes.js I will have a look

talmax1124 commented 3 years ago

@Slakie

import path from "path"; import express from "express"; import dotenv from "dotenv"; import cloudinary from "cloudinary"; import { CloudinaryStorage } from "multer-storage-cloudinary"; import multer from "multer";

dotenv.config();

const cloud = cloudinary.v2; const router = express.Router();

cloud.config({ cloud_name: process.env.CLOUDINARY_CLOUD_NAME, api_key: process.env.CLOUDINARY_API_KEY, api_secret: process.env.CLOUDINARY_API_SECRET, });

const storage = new CloudinaryStorage({ cloudinary: cloud, params: { folder: "cduoshoppic", // any desirable name transformation: 'media_lib_thumb', // the name you saved in transformation presets public_id: (req, file) => ${file.originalname.split(".")[0]}-${Date.now()}, }, });

function checkFileType(file, cb) { const filetypes = /jpg|jpeg|png/; const extname = filetypes.test( path.extname(file.originalname).toLocaleLowerCase() ); const mimetype = filetypes.test(file.mimetype); if (extname && mimetype) { return cb(null, true); } else { cb(null, false); } }

const mimetype = filetypes.test(file.mimetype);

if (extname && mimetype) { return cb(null, true,); } else { cb(null, false); }

const upload = multer({ storage,

fileFilter: function (req, file, cb) { checkFileType(file, cb); }, });

router.post("/", upload.single("image"), (req, res) => { res.send(req.file.path); });

export default router;

talmax1124 commented 3 years ago

@Slakie Can I have the link to your repo? Maybe if I copy that, it can work since its working for you

talmax1124 commented 3 years ago

@Slakie Here is my repo link: https://github.com/talmax1124/shop

Slakie commented 3 years ago

@talmax1124 remove everything from line 42 to 48

Slakie commented 3 years ago

@talmax1124 it is code duplication from line 42 to 48, have you fixed it? Is it all working now?

talmax1124 commented 3 years ago

Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'cloudinary' imported from /Users/carlosdiazplaza/Documents/GitHub/shop/backend/routes/uploadRoutes.js

@Slakie

talmax1124 commented 3 years ago

@Slakie & I have re-installed it already

Slakie commented 3 years ago

@talmax1124 have you installed two dependencies "cloudinary" and "multer-storage-cloudinary" I mentioned in the post above?

talmax1124 commented 3 years ago

Yes I Have @Slakie

Slakie commented 3 years ago

@talmax1124 I don't get it, it is giving you error or not anymore?

talmax1124 commented 3 years ago

I got it! I just installed some modules seperately and did some npm updates and it works now I think. I am going to check now

talmax1124 commented 3 years ago

It Worksssssss. YAY. Thank you so much @Slakie

Slakie commented 3 years ago

@talmax1124 That's great to hear, glad I could help πŸ˜ƒ

talmax1124 commented 3 years ago

Have you implemented some other features? I implemented phone number but perhaps can you help me on just this part. I want to let users sign in with either phone number or email address. How would I do that for the login? You can check my website and login with one of the john doe account and you will see I have implemented phone number but I can't get phone number to work for sign in. Can you help with that? @Slakie

Slakie commented 3 years ago

@talmax1124 sorry, I have not implemented any other features... I strongly believe you can sort that out on your own... It will be more helpful that way πŸ˜ƒ good luck!

aboubacarmsr commented 3 years ago

@talmax1124 sorry, I have not implemented any other features... I strongly believe you can sort that out on your own... It will be more helpful that way good luck!

I know this is a little bit late but using your code duplicates the image on upload on Cloudinary and i can't figure out why. Can you please help ?

talmax1124 commented 3 years ago

What do you mean duplicating? @aboubacarmsr

talmax1124 commented 3 years ago

Also @aboubacarmsr do you know how to add order status to frontend in orderscreen? I already have most of backend ready and done.