Skip to main content

Posts

SystemD System-V (init.d) compatibility layer and auto-restarts

SystemD (eg. RHEL7) has a compatibility layer for init.d services. When you run '/etc/init.d/ start', it doesn't do what you think it does; there is some magic involved. Have a look at the TurnKey Linux blog for some insight around that.

One of the great things about System-D is that you can auto-restart failed services. In this post, I want to show you how you take a init.d script (provided via a third-party package), and add some System-D goodness for making sure it can restart on failure.

For this example, I shall be using Globus's GridFTP service, which I have been working with recently. I found that after some routine patching, perhaps due to other patching activity around DNS, failed to resolve some name and failed to start the service.

SystemD will start (by 'magic' compatibility layer) init.d services when the name of the service doesn't exist (eg. systemctl restart globus-gridftp-server.service will auto-resolve to /etc/rc.d/init.d/globus-gridftp-se…
Recent posts

Confluent Schema Registry, cURL and Manipulating JSON with jq

Confluent Schema Registry, cURL and Manipulating JSON with jq It's been a while since I last posted, but today I figured out how to do something useful. I've been deploying some Kafka recently, and today was playing with Schema Registry and AVRO. AVRO is a binary format with a schema defined in JSON, so a schema looks something like this:
{   "name": "my_thingy",   "doc":"Initial demonstration of schema",   "type": "record",
  "fields": [     {       "name":"submission_time",       "type":"long",       "logical-type":"timestamp-millis",       "doc":"When first seen. ms since Unix epoch"     },     {       "name":"submitted_from",       "type":"string",       "doc":"Source Hostname or IP at entry of log submission"     } }
Nothing terribly interesting, and this post is not about AVRO. In…

Deploying with Ansible when Outgoing Access to Internet is Unavailable (BYO proxy)

So, you've completed your Dev environment in some nice throwaway VMs on your workstation (perhaps using Vagrant); your Ansible playbook is ready with a nice sheen to it, and your new shiny VMs are ready and just begging to receive instruction from the playbook you have so lovingly crafted [over the past few weeks, and expect to deploy to Test and Prod in a matter of a few days].

... wait, what you mean I don't have outgoing internet access; how am I supposed to get to my third-party repositories etc. Is there a proxy I can use at least... not available in the DMZ; I see ...

These sort of struggles are common as soon as you leave the nice comfortable confines of your development environment (where outgoing access is often easy) and start entering the real world of your network.

[Moral: don't expect initial deployments to Test/QA to be terribly easy; there will be gotchas if its a different environment to Dev. Test should however be the same as Prod.]

So we need to do someth…

Laptop with small screen + Hyper-V + Install CentOS

So I've got a nice new Lenovo ThinkPad laptop for work. Its got lots of memory and disk, awesome battery life, and it's nice and portable; just the thing for a systems engineer -- it even has a wired ethernet connection (I was previously using a Mac Air). It doesn't have a very large resolution though, just 1366 x 768.

Normally I would use VirtualBox, but on this laptop I'm trying out Hyper-V (because that's what Docker for Windows requires on Windows 10).

I went to install CentOS 7 (7.4), but the resolution of the default framebuffer is larger than can fit on my host's screen. This biggest annoyance about this is that Hyper-V wasn't showing me scroll-bars, and it also only allows to zoom in (for Hi-DPI screens), and not to zoom out. Being able to zoom out was very useful feature in VirtualBox, but the biggest problem was that it cut off the bottom of the screen, so I couldn't proceed with the install. Even text-mode installation was impractical because…

Provisioning Limited Access via Squid Proxy to Docker Containers

You have a Docker environment in your network; you limit outgoing traffic from your servers using a whitelisted proxy configuration (Squid) which can then be used for auditing. You wish to allow some containers access to some sites; perhaps even allow some containers access to any site.

How then, can you grant access based on the container? You need to add an extra level of authentication.

You could set up user-credentials for each container that needs access, but that is an administrative burden, and further a lot of software doesn't work well with a proxy that needs user authentication, which is a support concern.

Or we could use ident lookups to determine the 'user', which can then be used to evaluate ACL conditions. To do that, we would need to set up some specialised 'identd' type service on our docker host(s) that was able to return the container name associated with a given source-dest TCP port pair.

This turns out to be entirely feasible, and has the added …

Using Ansible to Fix CFEngine (after a trust failure as a result of re-addressing policy hub)

One of the things I have been given (cursed with) in my life in IT is maintenance of CFEngine. CFEngine is one of the oldest, typically left on the wayside, systems for configuration management on Linux and other Unix (and I think also Windows these days).... I'd love to drastically refactor and clean it up, because its grown pretty organically.

Anyway, its on some old equipment and I need to migrate it to a new IP to keep management happy from a risk perspective. How bad could that get? Turns out that CFEngine is sensitive to this, and I get another deluge of email:
!! Not authorized to trust the server=XXX.example.com's public key (trustkey=false)  !! Authentication dialogue with XXX.example.com failed  ... ad nuseum
So.... I think I just broke all the CFEngine agents, which won't be able to grab policy updates. Let's fix this using Ansible.

VirtualBox 5.1.27 + RHEL 7.4 (and others) + kernel update = suggest double reboot

For a long time now, every damn time I go to apply a kernel update, I have to rebuild the VirtualBox Guest Additions. If I have any vboxsf mounts set to mount at boot in /etc/fstab, I can look forward to a rescue-mode prompt. This is not specific to RHEL7, or to RHEL (I see plenty of reports of Ubuntu 11.04 with the same issue).

The 'dkms' (Dynamic Kernel Module Support) is meant to prevent this issue by triggering a rebuild when a kernel package is updated. It's installed,....  so why isn't it working. Time to get my hands dirty and learn a bit about dkms.