Fix annoying max open files for Emacs

I don't know which version to start with, the max open files of macOS has been changed to 1024, which is too small to develop with LSP in Emacs. And this problem cannot be simply solved by increasing ulimit. In this reddit thread, rpluim user mentioned:

Emacs uses pselect, which is limited to FD_SETSIZE file descriptors, usually 1024. I suspect you've got one of the file-watching utilities enabled in emacs, which tends to use up a lot of file descriptors.

…

Increasing the maxfiles limit will not change the value of FD_SETSIZE compiled into emacs and the macOS libraries. Emacs would have to move to using poll or kqueue to fully solve this issue.

This information can also be found in the macOS developer documentation:

The default size FD_SETSIZE (currently 1024) is somewhat smaller than the current kernel limit to the number of open files. However, in order to accommodate programs which might potentially use a larger number of open files with select, it is possible to increase this size within a program by providing a larger definition of FD_SETSIZE before the inclusion of <sys/types.h>.

But the document doesn't mention how to change it. I searched and found a similar problem with erlang with solution:

CFLAGS="-DFD_SETSIZE=10000 -DDARWIN_UNLIMITED_SELECT"

Great! After some tests, the max open files in Emacs is successfully changed to 10000. Here are the steps:

1. To increase the system-level ulimit limit. Concrete steps can be found in this gist

2. Compile Emacs from source, specifying the CFLAGS parameter when configure.

   git clone https://git.savannah.gnu.org/emacs && cd emacs
   git checkout emacs-28
   
   ./autogen.sh
   ./configure "CFLAGS=-DFD_SETSIZE=10000 -DDARWIN_UNLIMITED_SELECT"
   
   export CPATH=`xcrun --show-sdk-path`/usr/include:`xcrun --show-sdk-path`/usr/include/libxml2
   
   make -j 4 && make install

3. Test the newly compiled Emacs

   (shell-command-to-string "ulimit -n")
   ;; 10000
   
   (dotimes (i 2000)
    (make-process
     :name (format "Sleep-%s" i)
     :buffer nil
     :command '("sleep" "60000")
     :connection-type 'pipe))

   The dotimes block will create 2000 processes, and we can use lsof -p ${emacs_pid} | wc -l to check this. It should be more than 4000, since every process will open two files: stdout and stderr.

I hope this can help you to overcome this annoying issue when hack in Emacs.

Note: Although this is tested on macOS, other OS should share similar steps.

Update: as pointed out by reddit user amake, there is a patch attempt to replace select with poll, and increase max fd to 10240. Although it's not merged in master, users interested can try feature/more-fds branch, report bug if you have, hope this issue can be fixed in upstream.

Discussions on Lobster and r/emacs.