The File Melting Multer With Nodejs And Express

Aug 18, 2021 | 8 min read | 2460 Views | Comments

Multer is a mature multipart/form-data supreme middleware for Node.js. Its been there for a long time, already explored and discussed by many elite developers. In this article, we will revisit the core capabilities of the Multer and will explore scenarios like single/multiple file upload using forms or AJAX.

Album Post:

For quick read. Checkout this amazing album!

Working with Multer feels like a bar of melting chocolate; very soothing. This package is handling the horror of multipart/form-data gracefully. After working along with Multer and doing file upload operations I came across common scenarios which are revolving around the internet related to file upload.

These operations can be classified as follows:

File Upload Using Forms (Single/Multiple)
File Upload Using AJAX (Single/Multiple)

Before starting with the above kind of operations, we will first check the initial setup of Multer.

Initializing Project

After creating an empty folder and running “npm init” command; proceed with executing the following commands to install necessary packages. Check out more about multer package.

npm i express
npm i multer
npm i uuid
npm i nodemon -D

The project structure will be as follows:

multer-api
|-- node_modules
|-- public
    |-- assets
        |-- css
            |-- style.css
        |-- js
            |-- script.js
	|-- index.html
|-- routes
	|-- upload.js
|-- uploads
|-- index.js
|-- multer.js
|-- package.json

Starting NodeJS API

Start your API server with following boilerplate code:

const express = require('express')

const app = express()

// static files
app.use(express.static('public'))
app.use('/assets', express.static('/public/assets'))
app.use('/uploads', express.static('uploads'))

app.get('/', function (req, res) {
    res.sendFile('index.html')
})

app.use('/', require('./routes/upload'))

app.listen(3000, () => console.log('Server is running on 3000'))

Creating the UI

Create index.html inside the public folder; the index file will contain the UI for this Multer app along with assets like style.css and script.js.

<!doctype html>
<html lang="en">

<head>

    <title>Multer REST API</title>

    <!-- meta -->
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

    <!-- css -->
    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.0/dist/css/bootstrap.min.css" rel="stylesheet"
        integrity="sha384-KyZXEAg3QhqLMpG8r+8fhAXLRk2vvoC2f3B09zVXn8CA5QIVfZOJ3BCsw2P0p/We" crossorigin="anonymous">
    <link rel="stylesheet" href="assets/css/style.css">
</head>

<body>
    <div class="container my-5">
        <div class="row">
            <div class="col">
                <h1 class="mb-4 text-center pb-2">
                    ?? <span class="text-primary">Multer</span> REST API
                </h1>
            </div>
        </div>
    </div>

    <!-- js -->
    <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.1.0/dist/js/bootstrap.bundle.min.js"
        integrity="sha384-U1DAWAznBHeqEIlVSCgzq+c9gqGAJn5c/t99JyeKa9xxaYpSvHU5awsuZVVFIhvj"
        crossorigin="anonymous"></script>
    <script src="assets/js/script.js"></script>
</body>

</html>

Setting up the API routes for Upload

Create a file inside routes folder named upload; this file will hold up the endpoints for the file upload and file handling operations.

const express = require('express')
const fs = require('fs')
const upload = require('./../multer')

const router = express.Router();

module.exports = router;

Setting up the Multer

Create a multer.js file at the root level, this file will hold the Multer configuration which we will be used throughout the application.

const multer = require('multer')
const uuid = require('uuid')

const storage = multer.diskStorage({
    destination: function (req, file, cb) {
        cb(null, 'uploads')
    },
    filename: function (req, file, cb) {
        const nameSplit = file.originalname.split('.');
        const extension = nameSplit.length ? nameSplit[nameSplit.length - 1] : '';
        cb(null, `${uuid.v4()}.${extension}`)
    }
})
// const upload = multer({ dest: 'uploads' });
const upload = multer({ storage });

module.exports = upload;

Here we are importing the Multer and UUID packages for the setup. Multer provides a method named diskStorage() to configure the storage as per our own convenience. This method takes an object with destination and filename properties as functions. In the destination function, we simply need to pass the folder where we will going to store the uploaded files. The next property filename simply allows us to name the file as per our code logic demands; we are making use of the UUID package to name the file uniquely with appropriate extensions.

Now lets bring back the main goal to achieve and start coding for the same:

File Upload Using Forms (Single/Multiple)
File Upload Using AJAX (Single/Multiple)

1. File Upload Using Forms (Single/Multiple)

First we will create the endpoint for uploading files using the HTML form tag. We will make use of the routes/upload.js file to create the endpoints as follows:

router.post('/upload-form-file', upload.single('avatar'), function (req, res, next) {
    res.redirect('/?message=File uploaded');
})

router.post('/upload-form-files', upload.array('avatar'), function (req, res, next) {
    res.redirect('/?message=Files uploaded');
})

Pretty straightforward; we have made 2 endpoints for the file upload using HTML form. One is for single and the other one is for multiple file upload. These can be achieved by using the Multer object's single and array methods. The name provided here is an avatar (some random name) which has to be same as the name used for the file upload control. Additionally, once uploaded we are redirecting to the index.html page only with some query strings attached which can be used on the UI to prompt the file upload success. The Multer is attached as a middleware to your actual endpoint.

Now let us create the UI to upload the files using the HTML form's POST method and a provided action path. One must have to add the enctype="multipart/form-data" to the form tag; to make it work with Multer endpoint.

<!-- forms -->
<h6 class="mb-3 text-muted text-center text-uppercase">Using <span class="text-success">Forms</span></h6>
<div class="row mb-3">
	<div class="col-md-6">
		<div class="card mb-3">
			<div class="card-body">
				<h6 class="card-title">Single File Upload</h6>
				<form action="http://localhost:3000/upload-form-file" method="POST"
					enctype="multipart/form-data">
					<div class="mb-2">
						<input type="file" class="form-control form-control-sm " name="avatar">
						<div class="form-text">Select a single file.</div>
					</div>
					<button type="submit" class="btn btn-success btn-sm">Submit</button>
				</form>
			</div>
		</div>
	</div>
	<div class="col-md-6">
		<div class="card mb-3">
			<div class="card-body">
				<h6 class="card-title">Multi File Upload</h6>
				<form action="http://localhost:3000/upload-form-files" method="POST"
					enctype="multipart/form-data">
					<div class="mb-2">
						<input type="file" class="form-control form-control-sm " name="avatar" multiple>
						<div class="form-text">Select multiple files.</div>
					</div>
					<button type="submit" class="btn btn-success btn-sm">Submit</button>
				</form>
			</div>
		</div>
	</div>
</div>
<!-- /forms -->

Here we are setting up the action path to the respective enpoints and with the method POST. Also we are setting the name attribute of the file input to "avatar" same as that of the API. Most importantly is to set the enctype attribute to "multipart/form-data".

2. File Upload Using AJAX (Single/Multiple)

We will add the endpoints for uploading files using the AJAX call using the Fetch API. We will make use of the routes/upload.js file to create the endpoints as follows:

routes/upload.js

router.post('/upload-file', upload.single('avatar'), function (req, res) {
    res.json({
        message: 'File uploaded',
        files: req.file
    });
})

router.post('/upload-files', upload.array('avatar'), function (req, res) {
    res.json({
        message: 'Files uploaded',
        files: req.files
    });
})

The basics remain the same as that of form endpoint; only the difference here is the name of the endpoint and the response as JSON. We are sending a JSON object with a message and file object which denotes the file uploaded. Once the Multer middleware completes the file uploading operation it attaches the file to the request object. In the case of single file upload, we will get file object and for multiple, we get files object.

Now let us create the UI to upload the files using the AJAX with Fetch API.

public/index.html

<!-- ajax -->
<h6 class="mb-3 text-muted text-center text-uppercase">Using <span class="text-danger">AJAX</span></h6>
<div class="row mb-3">
	<div class="col-md-6">
		<div class="card mb-3">
			<div class="card-body">
				<h6 class="card-title">Single File Upload</h6>
				<div class="mb-2">
					<input type="file" class="form-control form-control-sm " id="file">
					<div class="form-text">Select a single file.</div>
				</div>
				<button id="btnSingle" class="btn btn-danger btn-sm">Submit</button>
			</div>
		</div>
	</div>
	<div class="col-md-6">
		<div class="card mb-3">
			<div class="card-body">
				<h6 class="card-title">Multi File Upload</h6>
				<div class="mb-2">
					<input type="file" class="form-control form-control-sm " id="files" multiple>
					<div class="form-text">Select multiple files.</div>
				</div>
				<button id="btnMultiple" class="btn btn-danger btn-sm">Submit</button>
			</div>
		</div>
	</div>
</div>
<!-- ajax -->

Nothing fancy here, just some plain HTML code. The party is going on inside the script.js file.

public/assets/js/script.js

const baseUrl = 'http://localhost:3000/';

// controls
const btnSingle = document.getElementById('btnSingle');
const btnMultiple = document.getElementById('btnMultiple');
const fileInput = document.getElementById('file');
const filesInput = document.getElementById('files');

// event listeners
btnSingle.addEventListener('click', function (e) {
    const formData = new FormData();
    formData.append('avatar', fileInput.files[0]);
    fileInput.value = '';

    fetch(`${baseUrl}upload-file`, {
        method: 'POST',
        body: formData
    })
        .then(res => res.json())
        .then(data => alert(`Using AJAX: ${data.message}`));
});
btnMultiple.addEventListener('click', function (e) {
    const formData = new FormData();
    Object.values(filesInput.files)
        .forEach(file => formData.append('avatar', file));
    filesInput.value = '';

    fetch(`${baseUrl}upload-files`, {
        method: 'POST',
        body: formData
    })
        .then(res => res.json())
        .then(data => alert(`Using AJAX: ${data.message}`));
});

Here we created the controls followed by an event listener for the button click. Lastly, we are making call to the endpoints when click event occurs with attached form data that holds the file. For single file we have to use fileInput.files[0] in order to obtain a single file; but for multiple files we need to iterate through the filesInput.files object values and append each file to the form data. The rest of the code remains the same of both single and multiple files.

Pretty much that's it for the file uploading part. Let us make our app a little bit more interactive by additionally adding a display file functionality to show on the UI how many files are there in the upload folder and also allow users to remove files they want to.

Show and Remove File

These operations are not dependent on Multer, instead, this is just to complete the cycle of uploading and showing the files those are been uploaded. We are making use of the "fs" package of Node.js to perform the file handling operations. Let us create the endpoint one by one:

Adding the API endpoints

We are adding these endpoint to the same routes/upload.js file.

Get Files (routes/upload.js)

router.get('/get-files', function (req, res) {

    const files = fs.readdirSync('uploads');
    const stats = files.map(file => ({
        name: file,
        ...fs.statSync(`uploads/${file}`)
    })).sort((a, b) => b.ctimeMs - a.mtimeMs);

    res.json(stats.map(i => i.name));
})

In order to get the files that are present inside the upload directory, we have to use the readdirSync method (Note: We can use the non-sync method too but for simplicity, I am making use of sync methods). The catch with the readdirSync method is that it doesn't provide file names in their creation order. So to obtain files based on their creation sequence we are mapping the files with the statSync method which provides additional information for a file and hence we are sorting based on the mtimeMs which is the last modification time.

Remove a File (routes/upload.js)

router.get('/remove-file/:fileName', function (req, res) {

    const fileName = req.params.fileName;
    fs.unlinkSync(`uploads/${fileName}`);

    res.json({
        message: 'File removed',
    });
})

There is a method named unlinkSync which can be used to remove files based on the provided path. We need to simply pass the file name to remove any file from the upload folder.

Remove Files (routes/upload.js)

router.get('/remove-files', function (req, res) {

    const files = fs.readdirSync('uploads');
    files.forEach(fileName => fs.unlinkSync(`uploads/${fileName}`));

    res.json({
        message: 'Files removed',
    });
})

Here we are getting all the file names using the same readdirSyn method and the using iteration to remove files using the unlinkSync method.

Creating the UI

Adding the below HTML divs after the AJAX UI. The CSS for the UI is added in the style.css file

<!-- files -->
<div class="mb-3">
	<button class="btn btn-outline-danger btn-sm float-end" onclick="removeFiles()">Remove All</button>
	<h5>Uploads Folder:</h5>
</div>
<div class="files"></div>
<!-- /files -->

Making the AJAX calls

// functions
function isImage(fileName) {
    const [name, extension] = fileName.split('.');
    return ['png', 'jpg', 'gif'].includes(extension.toLowerCase());
}

function getFiles() {
    fetch(`${baseUrl}get-files`)
        .then(res => res.json())
        .then(data => {
            let html = '';
            data.forEach(fileName => {
                html += `
                    <div>
                        <a href="uploads/${fileName}" download>
                            ${
                                isImage(fileName) ? 
                                    `<img src="uploads/${fileName}" alt="${fileName}">` : 
                                    `?? <span>${fileName}</span>`
                            }
                        </a>
                        <button class="btn btn-sm btn-secondary" onclick="removeFile(this, '${fileName}')">Remove</button>
                    </div>
                `;
            });
            dvFiles.innerHTML = html;
        })
}

function removeFile(e, fileName) {
    const result = confirm(`Are you sure you want to remove ${fileName}?`);
    if (result) {
        fetch(`${baseUrl}remove-file/${fileName}`)
            .then(res => res.json())
            .then(_ => e.parentElement.remove())
    }
}

function removeFiles() {
    const result = confirm(`Are you sure you want to remove all files?`);
    if (result) {
        fetch(`${baseUrl}remove-files`)
            .then(res => res.json())
            .then(_ => dvFiles.innerHTML = '')
    }
}

We have created different functions to handle these many operations. The getFiles function brings the files and forms the HTML to display on the page. The removeFile and removeFile functions are for handling the remove operations. We will call the getFiles function on the initial load and when the user uploads the file using AJAX.

Git Repository

Check out the git repository for this project or download the code.

Download Code Git Repository

Summary

Understanding and gaining full control over multipart/form-data is quite tricky and required some maturity to handle. Multer makes life easier for handling multipart/form-data and lets you focus on your code logic instead of some common file upload operation. There is much more stuff Multer is capable of doing but that is a story for another day. In this article, we implemented the common ways Multer used in any real-world application.

Hope this article helps.