MongoDB "Too Many Open Files"? Raise the limit

October 20, 2010 | Tags: mongodb

This blog post is intended to supplement the "Too Many Open Files" page in the mongoDB docs.

Raising the file limit for MongoDB

If you installed from the Ubuntu/Debian package, then there is a simple way to increase the open file limit. MongoDB's startup script is /etc/init/mongodb.conf. This is an upstart script which supersedes the /etc/init.d scripts we're all use to. All you need to do is add "ulimit -n {X}" to the script block before mongodb is launched, replacing X with the limit you want (I use 65000). That sets the ulimit for the script and any processes it launches (therefore mongodb). Here is an example /etc/init/mongodb.conf

# Ubuntu upstart file at /etc/init/mongodb.conf

pre-start script
  mkdir -p /db/mongodb/
  mkdir -p /var/log/mongodb/
end script

start on runlevel [2345]
stop on runlevel [06]

script
  ulimit -n 65000

  ENABLE_MONGODB="yes"
  if [ -f /etc/default/mongodb ]; then . /etc/default/mongodb; fi
  if [ "x$ENABLE_MONGODB" = "xyes" ]; then
    exec start-stop-daemon --start --quiet --chuid mongodb --exec  /usr/bin/mongod -- --config /etc/mongodb.conf
  fi

end script

Once mongoDB is up, you can check the limit for the process by doing cat /proc/{pid}/limit. Replace {pid} with the pid of mongoDB. To get the pid, just do "ps axu | grep mongodb".

If you aren't using the install packages, then you'll need to add the ulimit command to your own startup script. Keep in mind that ulimit is a bash command, not a regular binary tool, so look at "man bash" for more info on ulimit.

This blog post suggests that you can set system wide limits by editing /etc/security/limits.conf and /etc/pam.d/common-session, however, this only applies interactive and non-interactive shells (and processes started by them). When using just this technique, it didn't appear to affect the open file limit of the mongodb process started by the default upstart script (without my added ulimit statement).

If you want to try system wide limits, then add a line like the following to /etc/security/limits.conf:

*  -  nofile 65000

See "man limits.conf" for more info.

In /etc/pam.d/common-session, enable the limits module by adding:

session required pam_limits.so

Keep in mind that all this really does is have PAM set the limit for interactive and non-interactive shells when loaded. Processes then pickup these limits when started from shells/scripts. You should probably do a restart to fully enact these settings. If someone out there gets system wide settings to apply to mongoDB, let me know with a comment.

Update:

It occurred to me after writing this post that you should put "ulimit -n 65000" in /etc/default/mongodb. If that file doesn't exist, create it. This is the proper place for it. As you can see in the upstart file, it will run /etc/default/mongodb if it exists before it launches mongodb.

Andrew Rollins

MongoDB "Too Many Open Files"? Raise the limit