How To Solve : systemctl multiple failed services in Centos 7

Solved ! systemctl multiple failed services.  I have a box running CentOS 7.2 kernel 3.10. selinux is disabled. I have several services failing on bootup. I have no idea what to do about any of them. I've been googling for several hours now and haven't been able to figure anything out.

How To Solve : systemctl multiple failed services in Centos 7

CODE: 

[root@localhost log]# systemctl --failed                                                       
  UNIT                     LOAD   ACTIVE SUB    DESCRIPTION                                       
<E2><97><8F> auditd.service           loaded failed failed Security Auditing Ser
<E2><97><8F> iptables.service         loaded failed failed IPv4 firewall with ip
<E2><97><8F> kdump.service            loaded failed failed Crash recovery kernel
<E2><97><8F> network.service          loaded failed failed LSB: Bring up/down ne
<E2><97><8F> systemd-networkd.service loaded failed failed Network Service
<E2><97><8F> systemd-networkd.socket  loaded failed failed networkd rtnetlink so

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

6 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.

CODE: 

systemctl status auditd.service
��● auditd.service - Security Auditing Service
   Loaded: loaded (/usr/lib/systemd/system/auditd.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2011-05-10 00:00:43 UTC; 46min ago
  Process: 272 ExecStartPost=/sbin/augenrules --load (code=exited, status=0/SUCCESS)
  Process: 270 ExecStart=/sbin/auditd -n (code=exited, status=6)
 Main PID: 270 (code=exited, status=6)

May 10 00:00:43 localhost augenrules[272]: enabled 0
May 10 00:00:43 localhost augenrules[272]: flag 1
May 10 00:00:43 localhost augenrules[272]: pid 0
May 10 00:00:43 localhost augenrules[272]: rate_limit 0
May 10 00:00:43 localhost augenrules[272]: backlog_limit 320
May 10 00:00:43 localhost augenrules[272]: lost 0
May 10 00:00:43 localhost augenrules[272]: backlog 0
May 10 00:00:43 localhost systemd[1]: Failed to start Security Auditing Service.
May 10 00:00:43 localhost systemd[1]: Unit auditd.service entered failed state.
May 10 00:00:43 localhost systemd[1]: auditd.service failed.
Hint: Some lines were ellipsized, use -l to show in full.

CODE: 

systemctl status auditd.service
��● auditd.service - Security Auditing Service
   Loaded: loaded (/usr/lib/systemd/system/auditd.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2011-05-10 00:00:43 UTC; 46min ago
  Process: 272 ExecStartPost=/sbin/augenrules --load (code=exited, status=0/SUCCESS)
  Process: 270 ExecStart=/sbin/auditd -n (code=exited, status=6)
 Main PID: 270 (code=exited, status=6)

May 10 00:00:43 localhost augenrules[272]: enabled 0
May 10 00:00:43 localhost augenrules[272]: flag 1
May 10 00:00:43 localhost augenrules[272]: pid 0
May 10 00:00:43 localhost augenrules[272]: rate_limit 0
May 10 00:00:43 localhost augenrules[272]: backlog_limit 320
May 10 00:00:43 localhost augenrules[272]: lost 0
May 10 00:00:43 localhost augenrules[272]: backlog 0
May 10 00:00:43 localhost systemd[1]: Failed to start Security Auditing Service.
May 10 00:00:43 localhost systemd[1]: Unit auditd.service entered failed state.
May 10 00:00:43 localhost systemd[1]: auditd.service failed.
Hint: Some lines were ellipsized, use -l to show in full.

CODE:

systemctl status kdump.service                                                                             
��● kdump.service - Crash recovery kernel arming
   Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2011-05-10 00:01:28 UTC; 48min ago
  Process: 716 ExecStart=/usr/bin/kdumpctl start (code=exited, status=1/FAILURE)
 Main PID: 716 (code=exited, status=1/FAILURE)

May 10 00:01:28 localhost dracut[4482]: drwxr-xr-x   2 root     root        ...i
May 10 00:01:28 localhost dracut[4482]: lrwxrwxrwx   1 root     root        ...k
May 10 00:01:28 localhost dracut[4482]: lrwxrwxrwx   1 root     root        ...n
May 10 00:01:28 localhost dracut[4482]: ====================================...=
May 10 00:01:28 localhost kdumpctl[716]: No memory reserved for crash kernel.
May 10 00:01:28 localhost kdumpctl[716]: Starting kdump: [FAILED]
May 10 00:01:28 localhost systemd[1]: kdump.service: main process exited, c...RE
May 10 00:01:28 localhost systemd[1]: Failed to start Crash recovery kernel...g.
May 10 00:01:28 localhost systemd[1]: Unit kdump.service entered failed state.
May 10 00:01:28 localhost systemd[1]: kdump.service failed.
Hint: Some lines were ellipsized, use -l to show in full.

CODE:

systemctl status network.service
��● network.service - LSB: Bring up/down networking
   Loaded: loaded (/etc/rc.d/init.d/network)
   Active: failed (Result: exit-code) since Tue 2011-05-10 00:00:44 UTC; 50min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 329 ExecStart=/etc/rc.d/init.d/network start (code=exited, status=1/FAILURE)

May 10 00:00:44 localhost network[329]: Bringing up interface eth0:  ERROR  ....
May 10 00:00:44 localhost /etc/sysconfig/network-scripts/ifup-eth[594]: Devic...
May 10 00:00:44 localhost network[329]: [FAILED]
May 10 00:00:44 localhost network[329]: Bringing up interface eth1:  ERROR  ....
May 10 00:00:44 localhost /etc/sysconfig/network-scripts/ifup-eth[675]: Devic...
May 10 00:00:44 localhost network[329]: [FAILED]
May 10 00:00:44 localhost systemd[1]: network.service: control process exit...=1
May 10 00:00:44 localhost systemd[1]: Failed to start LSB: Bring up/down ne...g.
May 10 00:00:44 localhost systemd[1]: Unit network.service entered failed state.
May 10 00:00:44 localhost systemd[1]: network.service failed.
Hint: Some lines were ellipsized, use -l to show in full.

CODE: 

systemctl status systemd-networkd.service                                                                   
��● systemd-networkd.service - Network Service
   Loaded: loaded (/usr/lib/systemd/system/systemd-networkd.service; enabled; vendor preset: enabled)
   Active: failed (Result: start-limit) since Tue 2011-05-10 00:00:43 UTC; 51min ago
     Docs: man:systemd-networkd.service(8)
  Process: 351 ExecStart=/usr/lib/systemd/systemd-networkd (code=exited, status=226/NAMESPACE)
 Main PID: 351 (code=exited, status=226/NAMESPACE)

May 10 00:00:43 localhost systemd[1]: Unit systemd-networkd.service entered...e.
May 10 00:00:43 localhost systemd[1]: systemd-networkd.service failed.
May 10 00:00:43 localhost systemd[1]: systemd-networkd.service has no holdo...t.
May 10 00:00:43 localhost systemd[1]: start request repeated too quickly fo...ce
May 10 00:00:43 localhost systemd[1]: Failed to start Network Service.
May 10 00:00:43 localhost systemd[1]: Unit systemd-networkd.service entered...e.
May 10 00:00:43 localhost systemd[1]: systemd-networkd.service failed.
May 10 00:00:43 localhost systemd[1]: start request repeated too quickly fo...ce
May 10 00:00:43 localhost systemd[1]: Failed to start Network Service.
May 10 00:00:43 localhost systemd[1]: systemd-networkd.service failed.
Hint: Some lines were ellipsized, use -l to show in full.

CODE: 

systemctl status systemd-networkd.socket
��● systemd-networkd.socket - networkd rtnetlink socket
   Loaded: loaded (/usr/lib/systemd/system/systemd-networkd.socket; enabled; vendor preset: disabled)
   Active: failed (Result: service-failed-permanent) since Tue 2011-05-10 00:00:43 UTC; 51min ago
     Docs: man:systemd-networkd.service(8)
           man:rtnetlink(7)
   Listen: route 273 (Netlink)

May 10 00:00:43 localhost systemd[1]: Unit systemd-networkd.socket entered ...e.
Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
Hint: Some lines were ellipsized, use -l to show in full.

Here's what my grub statement looks like in grub,conf

CODE: 

title CentOS 7.2 x86_64
        root (hd0,0)
        kernel /vmlinuz.72 ro rootfstype=ext3 root=/dev/ram0 ramdisk_size=1200000 console=ttyS0,115200n8
        initrd /CentOS72.igz rdshell

I think I've managed to solve my Kdump service failure. After doing some research I learned that I could put crashkernel=256M into the kernel command at bootup and that would make kdump happy. However, I've also pretty much decided I don't want to use it. I removed kexec-tools from the install and restored my kernel command back to what it was. I no longer get the kdump.service failed status anymore.

I'm building the initramfs (or initrd) by downloading all the needed rpms for a minimal install, plus the development tools and the debugging package groups. I then execute the following commands to install the rpms into a new root folder: 

rpmdb --initdb --dbpath=./sms_rpms_22_DEC_2015/

rpm --root ./builddir --dbpath=./sms_rpms_22_DEC_2015/ --nosignature -ivh ./sms_rpms_22_DEC_2015/*.rpm

After that completes I then make any needed modifications (such as adding scripts, or creating user accounts and directories) to the OS within the ./builddir.

After that I then convert the .builddir into a filesystem by issuing the following:

mkfs.ext4 -m 0 -F -q -L Buildnumberx -b 1024 ./Buildnumberx.fs $RAMDISK_SIZE

After that I then convert it into the initramfs (CentOS72.igz) file I need using pigz. 

That might be more detail than you need but that's how the OS is getting built and installed on this machine. I believe the date in the Log is May 10 2011 is because that's a default date found in the Kernel.

I've made some progress though since I last posted. I've removed many of the customizations that were included in our previous build in CentOS 6.5 and now it's mostly a vanilla (minimal) install of CentOS 7.2.

I now only have three failed services. But I think it's the systemd-networkd.service and systemd-networkd.socket that I need most. 

I think I've managed to solve my Kdump service failure. After doing some research I learned that I could put crashkernel=256M into the kernel command at bootup and that would make kdump happy. However, I've also pretty much decided I don't want to use it. I removed kexec-tools from the install and restored my kernel command back to what it was. I no longer get the kdump.service failed status anymore.

The Kdump documentation found at https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/pdf/Kernel_Crash_Dump_Guide/Red_Hat_Enterprise_Linux-7-Kernel_Crash_Dump_Guide-en-US.pdf states:

Kdump is a kernel crash dumping mechanism that allows you to save the contents of the system's

memory for later analysis. It relies on kexec, which can be used to boot a Linux kernel from the

context of another kernel, bypass BIOS, and preserve the contents of the first kernel's memory that

would otherwise be lost.

In case of a system crash, kdump uses kexec to boot into a second kernel (a capture kernel). This

second kernel resides in a reserved part of the system memory that is inaccessible to the first kernel.

The second kernel then captures the contents of the crashed kernel's memory (a crash dump) and

saves it

I don't find this exactly necessary for my particular use case and would like to keep as much memory as possible free for use by other processes. So lastly, I'm still trying to get systemd-networkd working. I'm seeing mention of RTNetlink. I'm wondering if this is preventing systemd-networkd from coming up.

I've come to find that I don't need systemd-networkd and so I've removed it. That effectively solves all of my systemctl failed services.