Using Foreman to Manage Profile Based Applications

To run Kafka consumers in production, we can export the Profile to systemd or init that can be used to bring up as many instances as required. Following the readme of the foreman gem lead to many issues. The readme says we can use upstart, but I had to switch to systemd. Create a Procfile in the root of the Rails project.

consumer: bundle exec racecar PrintConsumer

In my case, I only have one process running. Run foreman in development:

pollywog@kafka:/src/messenger/rafk$ foreman start

You can see the output:

01:21:58 consumer.1 | started with pid 14812
01:21:58 consumer.1 | => Starting Racecar consumer PrintConsumer...
01:21:58 consumer.1 | => Detected Rails, booting application...
01:22:01 consumer.1 | => Wrooooom!
01:22:01 consumer.1 | => Ctrl-C to shutdown consumer
01:22:03 consumer.1 | I, [2019-02-07T01:22:01.126381 #14812]  INFO -- : New topics added to target list: test-top
01:22:03 consumer.1 | I, [2019-02-07T01:22:01.126432 #14812]  INFO -- : Fetching cluster metadata from kafka://kafka:9092

Once you test it on development, you can export to upstart like this:

foreman export upstart /etc/init
[foreman export] writing: app.conf
[foreman export] writing: app-consumer.conf
[foreman export] writing: app-consumer-1.conf

The exported files are in the /etc/init directory. The output of cat /etc/init/app.conf:

start on runlevel [2345]
stop on runlevel [!2345]

The output of cat /etc/init/app-consumer.conf:

start on starting app
stop on stopping app

The output of cat /etc/init/app-consumer-1.conf:

start on starting app-consumer
stop on stopping app-consumer
respawn

env PORT=5000

setuid app

chdir /src/messenger/rafk

exec bundle exec racecar PrintConsumer

Install upstart on Ubuntu.

sudo apt-add-repository ppa:canonical-foundations/upstart-daily
sudo apt-get update
sudo apt-get install upstart

This will not work in Debian releases 9 or above. We can use systemd. It comes with Ubuntu. You can check the version.

$ systemd --version
systemd 232
+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN
$ systemctl --version
systemd 232
+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN

We can export it to systemd:

sudo foreman export --app kafkapp --user pollywog systemd /etc/systemd/system/

kafkapp is my app name. pollywog is the user name to start the systemd.

The output:

[foreman export] writing: kafkapp-consumer@.service
[foreman export] creating: kafkapp-consumer.target.wants
[foreman export] symlinking: kafkapp-consumer.target.wants/kafkapp-consumer@5000.service -> ../kafkapp-consumer@.service
[foreman export] writing: kafkapp-consumer.target
[foreman export] writing: kafkapp.target

The commands:

systemctl start kafkapp.target

and

systemctl start kafkapp-consumer@.service

both gives error:

Failed to start kafkapp-consumer@.service: Unit name kafkapp-consumer@.service is missing the instance name.
See system logs and 'systemctl status kafkapp-consumer@.service' for details.
Failed to get properties: Unit name kafkapp-consumer@.service is missing the instance name.
systemctl start kafkapp.target

You can export to inittab.

 sudo foreman export inittab /etc/init/rc-sysinit.conf
[foreman export] writing: /etc/init/rc-sysinit.conf

You can see the contents of the generated file:

pollywog@kafka:/src/messenger/rafk$ cat /etc/init/rc-sysinit.conf
# ----- foreman app processes -----
AP01:4:respawn:/bin/su - app -c 'cd /src/messenger/rafk;export PORT=5000;bundle exec racecar PrintConsumer >> /var/log/app/consumer-1.log 2>&1'
# ----- end foreman app processes -----
pollywog@kafka:/src/messenger/rafk$ cat Procfile
consumer: bundle exec racecar PrintConsumer

References


Related Articles


Ace the Technical Interview

  • Easily find the gaps in your knowledge
  • Get customized lessons based on where you are
  • Take consistent action everyday
  • Builtin accountability to keep you on track
  • You will solve bigger problems over time
  • Get the job of your dreams

Take the 30 Day Coding Skills Challenge

Gain confidence to attend the interview

No spam ever. Unsubscribe anytime.