bradtraversy / proshop_mern

Shopping cart built with MERN & Redux
1.99k stars 1.18k forks source link

Images are uploaded to the server would eventually get deleted. #91

Closed Slakie closed 3 years ago

Slakie commented 3 years ago

Issue Link

Issue: Files are uploaded to the app but then disappear or are deleted after a while.

Resolution from Heroku support: The Heroku filesystem is ephemeral - that means that any changes to the filesystem whilst the dyno is running only last until that dyno is shut down or restarted. Each dyno boots with a clean copy of the filesystem from the most recent deploy. This is similar to how many container based systems, such as Docker, operate.

In addition, under normal operations dynos will restart every day in a process known as "Cycling".

These two facts mean that the filesystem on Heroku is not suitable for persistent storage of data. In cases where you need to store data we recommend using a database addon such as Postgres (for data) or a dedicated file storage service such as AWS S3 (for static files). If you don't want to set up an account with AWS to create an S3 bucket we also have addons here that handle storage and processing of static assets https://elements.heroku.com/addons

My resolution:

  1. Install 2 additional dependencies for the backend: "cloudinary" and "multer-storage-cloudinary"
  2. Sign up at cloudinary.com
  3. Copy your auto generated Environment variables from: cloudinary.com -> Dashboard
  4. Change your existing file code: backend -> routes -> uploadRoutes.js onto:
    import path from 'path';
    import express from 'express';
    import dotenv from 'dotenv';
    import cloudinary from 'cloudinary';
    import { CloudinaryStorage } from 'multer-storage-cloudinary';
    import multer from 'multer';

dotenv.config();

const cloud = cloudinary.v2; const router = express.Router();

cloud.config({ cloud_name: process.env.CLOUDINARY_CLOUD_NAME, api_key: process.env.CLOUDINARY_API_KEY, api_secret: process.env.CLOUDINARY_API_SECRET, });

const storage = new CloudinaryStorage({ cloudinary: cloud, params: { folder: 'folder_name', // any desirable folder name for your Media Library (uploaded images will be in this folder) public_id: (req, file) => ${file.originalname.split('.')[0]}-${Date.now()}, }, });

function checkFileType(file, cb) { const filetypes = /jpg|jpeg|png/; const extname = filetypes.test( path.extname(file.originalname).toLocaleLowerCase() ); const mimetype = filetypes.test(file.mimetype); if (extname && mimetype) { return cb(null, true); } else { cb(null, false); } }

const upload = multer({ storage, fileFilter: function (req, file, cb) { checkFileType(file, cb); }, });

router.post('/', upload.single('image'), (req, res) => { res.send(req.file.path); });

export default router;

basir commented 3 years ago

thanks for sharing the Cloudinary image upload feature. it is easy and straightforward. we have a plan to add AWS s3 bucket support for this project too.

Slakie commented 3 years ago

@basir no problems, sounds good looking forward to AWS S3 bucket...

niyon-x commented 3 years ago

@Slakie Did the images show up in homepage, but didn't show up in single product page or when searching the product?

Slakie commented 3 years ago

@SuneelGiri I am sorry, I didn't quite get your question?

niyon-x commented 3 years ago

@Slakie i faced issue similar to yours. But in my case the photos loaded in home screen, not in single product page, or after \category\:id routes. I fixed this my hosting my images in cloud.

Firefly08 commented 3 years ago

Hi @Slakie . I know that the issue is closed but maybe you can help me. Please. I had the same problem with the images... after a while, they are disappearing from Heroku. So I followed your steps and now on my Heroku app I have an error: 'Application error' and on the local version I have 'Request failed with status code 500'. My Heroku app: https://finalstore21.herokuapp.com/ My GitHub files: https://github.com/Firefly08/finalstore21 Can you help me, please?

Slakie commented 3 years ago

Hi @Firefly08,

It seems to me that the problem is that you are using you cloudinary api credentials directly in the uploadRoutes.js like this:

cloud.config({

cloud_name: process.env.dtqprzuw6,

api_key: process.env.773679569836399,

api_secret: process.env.oGtIKOfIbuSj4IpbuZVIUzSLxSQT,

});

Do this instead:

cloud.config({

cloud_name: “dtqprzuw6”,

api_key: “773679569836399”,

api_secret: “oGtIKOfIbuSj4IpbuZVIUzSLxSQT”,

})

Let me know if it works for you…?

On Wed, 18 Aug 2021 at 17:46, Firefly8 @.***> wrote:

Hi @Slakie https://github.com/Slakie . I know that the issue is closed but maybe you can help me. Please. I had the same problem with the images... after a while, they are disappearing from Heroku. So I followed your steps and now on my Heroku app I have an error: 'Application error' and on the local version I have 'Request failed with status code 500'. My Heroku app: https://finalstore21.herokuapp.com/ My GitHub files: https://github.com/Firefly08/finalstore21 Can you help me, please?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bradtraversy/proshop_mern/issues/91#issuecomment-901084932, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOSB2MBVEWAURTLON2CBW5LT5OTRHANCNFSM4V46MXMQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

talmax1124 commented 3 years ago

@Slakie Quick question, have you ever done a options feature?

Firefly08 commented 3 years ago

Hi @Firefly08, It seems to me that the problem is that you are using you cloudinary api credentials directly in the uploadRoutes.js like this: cloud.config({ cloud_name: process.env.dtqprzuw6, api_key: process.env.773679569836399, api_secret: process.env.oGtIKOfIbuSj4IpbuZVIUzSLxSQT, }); Do this instead: cloud.config({ cloud_name: “dtqprzuw6”, api_key: “773679569836399”, api_secret: “oGtIKOfIbuSj4IpbuZVIUzSLxSQT”, }) Let me know if it works for you…? On Wed, 18 Aug 2021 at 17:46, Firefly8 @.**> wrote: Hi @Slakie https://github.com/Slakie . I know that the issue is closed but maybe you can help me. Please. I had the same problem with the images... after a while, they are disappearing from Heroku. So I followed your steps and now on my Heroku app I have an error: 'Application error' and on the local version I have 'Request failed with status code 500'*. My Heroku app: https://finalstore21.herokuapp.com/ My GitHub files: https://github.com/Firefly08/finalstore21 Can you help me, please? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#91 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOSB2MBVEWAURTLON2CBW5LT5OTRHANCNFSM4V46MXMQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

Hi @Slakie, I changed the code, the site is working now, but the images are still not showing (eg. /uploads/image-1629230531199.png). I copied the link to the image from Cloudinary (eg. https://res.cloudinary.com/dtqprzuw6/image/upload/v1629289807/iphone-12_y88cbx.jpg) and this way is working. But I would like to use the normal upload not link to the image from Cloudinary.

Slakie commented 3 years ago

Hi @Firefly08, To be able to use the initial way, you would need to consider changing the hosting then, heroku using dynos, meaning after 30 mins of inactivity your space provided by heroku would be cleaned out so that means any uploads would be lost too. At your next activity heroku would always use the files that were initially deployed. However I am not sure if heroku paid tier would behave the same way…

On Thu, 19 Aug 2021 at 00:28, Firefly8 @.***> wrote:

Hi @Firefly08 https://github.com/Firefly08, It seems to me that the problem is that you are using you cloudinary api credentials directly in the uploadRoutes.js like this: cloud.config({ cloud_name: process.env.dtqprzuw6, api_key: process.env.773679569836399, api_secret: process.env.oGtIKOfIbuSj4IpbuZVIUzSLxSQT, }); Do this instead: cloud.config({ cloud_name: “dtqprzuw6”, api_key: “773679569836399”, api_secret: “oGtIKOfIbuSj4IpbuZVIUzSLxSQT”, }) Let me know if it works for you…? … <#m1255754356285647161> On Wed, 18 Aug 2021 at 17:46, Firefly8 @.**> wrote: Hi @Slakie https://github.com/Slakie https://github.com/Slakie . I know that the issue is closed but maybe you can help me. Please. I had the same problem with the images... after a while, they are disappearing from Heroku. So I followed your steps and now on my Heroku app I have an error: 'Application error' and on the local version I have 'Request failed with status code 500'*. My Heroku app: https://finalstore21.herokuapp.com/ My GitHub files: https://github.com/Firefly08/finalstore21 Can you help me, please? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#91 (comment) https://github.com/bradtraversy/proshop_mern/issues/91#issuecomment-901084932>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOSB2MBVEWAURTLON2CBW5LT5OTRHANCNFSM4V46MXMQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

I changed the code, the site is working now, but the images are still not showing (eg. /uploads/image-1629230531199.png). I copied the link to the image from Cloudinary (eg. https://res.cloudinary.com/dtqprzuw6/image/upload/v1629289807/iphone-12_y88cbx.jpg) and this way is working. But I would like to use the normal upload not link to the image from Cloudinary.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bradtraversy/proshop_mern/issues/91#issuecomment-901372931, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOSB2MHVHZIJVBELYQBFRITT5QCXJANCNFSM4V46MXMQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

Slakie commented 3 years ago

@Slakie Quick question, have you ever done a options feature?

Hi @talmax1124 , I am not sure which features you're on about... if we are talking about images transformation via cloudinary service yes I have.

talmax1124 commented 3 years ago

@Slakie not images. Like shirt size and something like that.

Slakie commented 3 years ago

@talmax1124 right, no I have not.