AIX BoF

Home Directories
I have not been able to have home directories automagically created when logging in with LDAP, I have added many of the user home dir's to the default AIX build but you may run into some that don't exist, you can easily make you home directory using sudo.
If you are adding LDAP to a AIX box with local accounts already established it is best practice to remove the account for LDAP users. Before you run this command you must MAKE SURE THAT SECLDAPCLNTD IS NOT RUNNING, if it is still running you will remove all of the admin accounts from LDAP!:
stop-secldapclntd && for i in ; do rmuser -p $i ; done

Here are two one liners to create all the current Unix, App, and DB admins home folders:



start-secldapclntd
for i in ; do
 mkdir /home/$i;
 chown -R $i. /home/$i;
done

for i in ; do
 mkdir /home/$i;
 chown -R $i.IRM /home/$i;
done



AIX 5.3 and 6.1

# Installing and configuring LDAP client service on AIX 5.3/6.1
Installation
All packages are available on the /media/software nfs share, instructions reference that location for installation.

# Install Java
If java is not currently installed you can install Java5 64 bit, this version is for AIX 5.3 AND 6.1:

 installp -acgXYd /media/software/AIX/java/ Java5_64.sdk
Once java is installed you may need to add it to your path, the following command will do this temporarily:

 export PATH=/usr/java5_64/bin:$PATH


# Install gskit
Source: https://www14.software.ibm.com/webapp/iwm/web/reg/pick.do?source=gskitupdt&S_PKG

 installp -acgXd /media/software/AIX/GSKIT/ gsksa.rte gskta.rte


# Install ITDS packages
Remove any previous version of ldap that are installed, if the following command returns with filesets they need to be removed through smit:

  lslpp -aL *ldap* # will list installed ldap packages


Source http://www-01.ibm.com/support/docview.wss?rs=767&context=SSPREK&uid=swg27009778#ver62
6.2 packages

 installp -acgXd /media/software/AIX/ITDS/6.2.0.2-TIV-ITDS-AIX-IF0002/images/ idsldap.clt32bit62 \
 idsldap.clt64bit62 idsldap.cltbase62 idsldap.msg62.en_US idsldap.clt_max_crypto32bit62 \
 idsldap.clt_max_crypto64bit62
Once those filesets are installed run 'lslpp -aL idsldap.*' to verify, output should be similar to this:

 lslpp -aL idsldap.*
  Fileset                      Level  State  Type  Description (Uninstaller)
  ----------------------------------------------------------------------------
  idsldap.clt32bit62.rte     6.2.0.n    C     F    Directory Server - 32 bit
                                                   Client
  idsldap.clt64bit62.rte     6.2.0.n    C     F    Directory Server - 64 bit
                                                   Client
  idsldap.clt_max_crypto32bit62.rte
                             6.2.0.n    C     F    Directory Server - 32 bit
                                                   Client (SSL)
  idsldap.clt_max_crypto64bit62.rte
                             6.2.0.n    C     F    Directory Server - 64 bit
                                                   Client (SSL)
  idsldap.cltbase62.adt      6.2.0.n    C     F    Directory Server - Base Client
  idsldap.cltbase62.rte      6.2.0.n    C     F    Directory Server - Base Client
  idsldap.msg62.en_US        6.2.0.n    C     F    Directory Server - Messages -
                                                   U.S. English (en)



# Configuration
methods.cfg
Paste the following into a root cli: (this assumes that you use TSM for backups and that the client is configured)

 if [ $(grep -c LDAP /usr/lib/security/methods.cfg) -eq "0" ];
  then
   echo "Backing up current methods.cfg, please wait"
   dsmc i /usr/lib/security/methods.cfg
   cat <>/usr/lib/security/methods.cfg

LDAP:
        program = /usr/lib/security/LDAP
        program_64 = /usr/lib/security/LDAP64
EOF
 else
   echo "Found LDAP stanza in /usr/lib/security/methods.cfg, assuming that it is correct."
 fi


CA Certs
Get certs from the repos

 cd /etc/security/ldap/ && wget .crt && wget .crt

Create and list key database for ssl, if you run into java errors check that your version of java is at least 1.4.2

 gsk7cmd -keydb -create -db /etc/security/ldap/key.kdb -pw -type cms
 gsk7cmd -cert -list CA -db /etc/security/ldap/key.kdb -pw

Add CA cert from to key database

gsk7cmd -cert -add -db /etc/security/ldap/key.kdb -file /etc/security/ldap/.crt -format ascii -label " CA cert" -pw -trust enable


# mksecldap command / ldap.cfg file
Run the mksecldap command to tie the system into LDAP, this may take a minute or more.
The mkseclsap command cannot be used to setup anonymous bind configurations, instead of using that command create the /etc/security/ldap/ldap.cfg using the following command:

cat </etc/security/ldap/ldap.cfg
ldapservers:,
authtype:ldap_auth
userattrmappath:/etc/security/ldap/2307user.map
groupattrmappath:/etc/security/ldap/2307group.map
userbasedn:
groupbasedn:
useSSL:yes
ldapsslkeyf:/etc/security/ldap/key.kdb
ldapsslkeypwd:
EOF


# IDS links
IDS needs certain libraries and binaries linked from /opt/IBM/ldap//... to /usr/lib, etc.
First step is to remove any current links

 /opt/IBM/ldap/V6.2/bin/idsrmlink -i -l 64 -s fullsrv
 /opt/IBM/ldap/V6.2/bin/idsrmlink -i -l 32 -s fullsrv

Second step is to create new links to the 32 bit binaries

 /opt/IBM/ldap/V6.2/bin/idslink -i -l 32 -s base


# Start secldap at reboot
Remove existing inittab entries

rmitab ldapclntd

Create new inittab entry

 mkitab 'ldapclntd:23456789:wait:/usr/sbin/start-secldapclntd  > /dev/console  > 2&>1


# /etc/security/user
Add ldap compatability to /etc/security/user, without this ldap login will not work

 chsec -f /etc/security/user -s default -a "SYSTEM=LDAP or compat"


# sudo
Add your sudo config, I haven't had success with the AIX Linux Toolbox sudo rpm, try the sudo-noldap package (also on AIX Linux Toolbox)

# Start secldap client
Run the following command to start the ldap client

start-secldapclntd








#
# Syslog
#

AIX pulls its logging configuration from /etc/syslog.conf, the standard AIX build as of May 2009 writes most messages to /var/adm/SYSLOG and does not include rotation. I like the logging to be more Linux-like so I append this to the end of the /etc/syslog.conf file, consider commenting out the existing logging if you don't want to write to /var/adm/SYSLOG:

 # Linux-ify the AIX logging setup and enable automagic rotation
 # Everything but mail and auth to messages
 *.info;mail.none;auth.none      /var/log/messages       rotate size 10m files 10 compress
 # Auth to secure
 auth.debug                      /var/log/secure         rotate size 10m files 10 compress
 # Mail to maillog
 mail.debug                      /var/log/maillog        rotate size 10m files 10 compress
 # Emergency messages to all users
 *.emerg         *ss
 *.info;mail.none                @

This will send:
All messages except mail and authentication to /var/log/messages.
All authentication messages to /var/log/secure.
All mail messages to /var/log/maillog.
All emergency messages to all users.
All messages except mail to

If you don't like seeing the "Message forwarded from hostname:" message all over splunk you need to run the following command line as root:
 chssys -s syslogd -a "-n"
All the files need to exist prior to syslog writing to them, it will not create them on its own:
 sudo touch /var/log/messages /var/log/secure /var/log/maillog
You must then restart syslog
 stopsrc -s syslogd && startsrc -s syslogd

Enabling console access on fully virt guests (virsh/virt-manager)

Using virsh or virt-manager you can gain access to the console by editing grub.conf, this allows you to watch the machine boot from a terminal.

Add the following to the kernel line in /etc/grub.conf and reboot:

console=tty0 console=ttyS0

Converting kvm guests from lvm to qcow2, base images and snapshots

lvm based kvm guests are fast but you lose some flexibility, playing with fedora/kvm on my laptop I prefer to use file based images. Converting from lvm images to qcow2 isn't hard but the documentation is sparse.

1. use qemu-img to convert from an lvm to qcow2 format:

qemu-img convert -O qcow2 /dev/vg_name/lv_name/ /var/lib/libvirt/images/image_name.qcow2

If you want the image compressed add '-c' right after the word convert.

2. edit the xml for the image

virsh edit image_name

modify the disk stanza, adding a type to the driver line; on the source line change 'dev' to 'file' and modify the path:

driver name='qemu' type='qcow2'
source file='/var/lib/libvirt/images/image_name.qcow2'

Creating images from with a base image allows quick rollouts of many boxes based on an single install - for example I have a 'golden image' of centos, I can stop that VM and create 2 servers using the original VM disk as a base file and writing changes to different files.

qemu-img create -b original_image.qcow2 -f qcow2 clone_image01.qcow2
qemu-img create -b original_image.qcow2 -f qcow2 clone_image02.qcow2

Taking this further I can then snapshot both images so once I start making changes, rolling back to a point in time prior to the changes is very easy:

qemu-img snapshot -c snapshot_name vm_image_name.qcow2

references:
http://www.linux-kvm.com/content/how-you-can-use-qemukvm-base-images-be-more-productive-part-1

DHCP hostnames

On my home LAN Windows and Ubuntu boxes push their hostnames into local DNS which makes them easily resolvable - my router lists the names associated with MACs in its web interface, this is one of those small things that is really convenient. For some reason my fedora boxes have not been doing this lately and I finally got annoyed enough to look it up, turns out the fix is very simple.

First off figure out which interface you are using, if you are using more then 1 then I will assume you know which one you want the hostname pushed from, easily enough you can run 'ipconfig' or 'ip addr show' and note the interface name. In most cases this will be eth0 for copper and something like wlan0 for a wireless connection. Once you have the interface name substitute it into the following command:

echo "DHCP_HOSTNAME=$HOSTNAME" >> /etc/sysconfig/network-scripts/ifcfg-

Then restart networking:

sudo service network restart

In the above example I am using '$HOSTNAME' which is the system hostname variable but you could set this to a static name if you desired.

Is Hyper Threading enabled?

Needed a quick way to check if Hyper Threading was enabled on some RHEL boxes, ended up writing a quick "script" that can be copied onto the command line.

I'll go through it line by line just for fun:

First we grab all lines matching "core id" from /proc/cpuinfo, sort them (in case the id's where not listed in numeric order), list the unique values and count them

cores=`grep "core id" /proc/cpuinfo|sort|uniq|wc -l`

Using grep I count the number of lines matching "processor" from /proc/cpuinfo

procs=`grep -c "processor" /proc/cpuinfo`

If we fine less cores the processors then Hyper Threading must be on

if [[ "$cores" -lt "$procs" ]]; then
echo -e "\n$HOSTNAME: cores=$cores, processors=$procs\n    HyperThreading: Enabled"

If we find the same number of processors and cores the Hyper Threading is off

elif [[ "$cores" -eq "$procs" ]]; then
echo -e "\n$HOSTNAME: cores=$cores, processors=$procs\n    HyperThreading: Disabled"

If neither case matches then we have run into a failure, or our math doesn't work on this particular box

else
echo "epic failure"
fi

And the whole thing...

cores=`grep "core id" /proc/cpuinfo|sort|uniq|wc -l`
procs=`grep -c "processor" /proc/cpuinfo`
if [[ "$cores" -lt "$procs" ]]; then
echo -e "\n$HOSTNAME: cores=$cores, processors=$procs\n    HyperThreading: Enabled"
elif [[ "$cores" -eq "$procs" ]]; then
echo -e "\n$HOSTNAME: cores=$cores, processors=$procs\n    HyperThreading: Disabled"
else
echo "epic failure"
fi

How I learned to Stop Worrying and Love the Bomb istat

I was recently tasked with organizing 137k+ small .jpg files into a folder structure based on year and quarter, why? users opening this directory with a ftp client complained that it took a "long time" to get a directory listing... apparently 15 - 20 minutes each time they opened the directory, honestly if a program didn't return anything in 15 minutes I would probably kill it and blame the server!

I really didn't think too much of the problem, in my head I though "i'll just use 'find' and 'stat'", which would have worked perfectly EXCEPT that I had to do this on a AIX 4.3 server and mounting the filesystem remotely was not an option.

A few problems with AIX 4.3 - no 'stat' command, in AIX 5.x you can install the coreutils rpm from the AIX toolbox to overcome this problem but you are up the creek without a paddle in 4.3! Also 'find' doesn't have all of the options you would usually have available on a newer version of linux - this was an issue in my case since I had to put the files into subdirectories (example: /basedirectory/2008/Q3) which meant that when searching for files to process in the basedirectory I did not want to descend into the yearly and quarterly subdirectories, easy with the -maxdepth option - which is not available in 4.3.

I ended up getting around the lack of -maxdepth in the find command by using the -prune option to remove subdirectories from processing, because the basedirectoy did not contain any directories except 200{8,9}/Q{1..4} this task was simplified even further by providing a common directory 'Q*'.

The lack of the 'stat' command had me banging my head against 'ls' for a day or so... The problem I have with 'ls' is in trying to get the year from 'ls -l', it works great for files older then 180 days but files less then 180 days are listed with the file modification timestamp in place of the year. I toyed with awk'ing the year/modifaction time column and checking if the value was an integer, which does work but you run into issues if your script is running within 180 days of the end of the year and examining files from the previous year, all the files will have timestamps which would cause you to examine the value of the current month vs. the month of the file being examined to determine the correct year - logic that I was uninterested in writing out.

Enter in my new most loved command in AIX: 'istat'
I was lucky enough to find a post mentioning 'istat' which "displays the i-node information for a particular file". 'istat' is simliar to the linux 'stat' command although it does not allow you modify the output using command line switches - nothing a little grep and awk won't fix! What 'istat' does do is handily format data about file creation, modification and access in an unambiguous matter - dates are always shown in the same format, unlike 'ls -l'. Without this tool I was writing a longer and longer script to deal with corner cases dealing with files modified 180 days ago and files modified around the last 3 months of the year - with 'istat' I was able to make my script much simpler and rely on the computer to hand me information in an consistent format.

I would be surprised if anyone has to solve this same problem but I will post the script anyways, as a warning this script is slow - 'istat' is not a tool for performance! Also working 'xargs' into the mix would make a more elegant solution in-place of 'find' and 'cat'.

In the following script I have disabled the actual move command - this will only print what would happen! uncomment the line beginning with 'mv' and it will move files.

#!/usr/bin/ksh
#
# organize files ending in $fileextension in $basedir
# by moving them into subdirectories $basedir/$year/$quarter
#


# backdate variable controls how many days old a file must be before
# it is considered for processing, 92 days is approx 3 months
# if you don't believe me ask google "3 months in days"
backdate=92

fileext=YOUR_FILE_EXTENTION
outfile=/tmp/jpg_organizer.out
basedir=YOUR_BASE_DIRECTORY

errors=0

# function to calulate which quarter a month lives in
calculate_quarter() {
case $month in
      Jan|Feb|Mar)
              quarter="Q1"
              ;;
      Apr|May|Jun)
              quarter="Q2"
              ;;
      Jul|Aug|Sep)
              quarter="Q3"
              ;;
      Oct|Nov|Dec)
              quarter="Q4"
              ;;
esac
}

# rudimentary error checking
error_check() {
let errors="$errors + $?"
if [[ $errors -gt 0 ]]; then
  echo "encountered an error, exiting"
  exit $?
fi
}

# find files older then $backdate and move them into $basedir/$year/$quarter directories
find $basedir -name Q\* -prune -o -name \*$fileext -mtime +$backdate -type f -print > $outfile
error_check
for i in `cat $outfile` ; do
filename=$i
fileattrib=`istat $i | grep "Last modified:"`
month=`echo $fileattrib | awk '{print $4}'`
year=`echo $fileattrib | awk '{print $7}'`
calculate_quarter
if [[ ! -d $basedir/$year/$quarter ]]; then
  mkdir -p $basedir/$year/$quarter
  error_check
fi
echo "moving:$filename to $basedir/$year/$quarter/"
#mv $filename $basedir/$year/$quarter/
error_check
done

rm $outfile

exit 0

AIX syslogd and splunk (and more)

AIX is what I would call a 'batteries not-included' OS; the vanilla DVD install leaves you with a functioning system that has telnet (with root access) enabled, no OpenSSL/OpenSSH, korn shell without autocomplete (must be enables 'set -o vi'), no logging, etc...
Since I work around a lot of RedHat boxes I tend to modify the AIX servers to have a simlar setup to RHEL, here are some of the steps I take:

Install the following rpm's from the aix toolbox:
bash (add /usr/bin/bash to /etc/security/login.cfg)
curl
coreutils
less
lsof
python
rsync
sudo
unzip
wget

Install OpenSSL and OpenSSH

Change root home directory to /root and change shell to bash:

mkdir /root && chuser home=/root shell=/usr/bin/bash root

Modify prompt for all users:

# Set bash prompt to be much more linux like
if [[ "$TERM" == "xterm" ]];then
if [[ "$SHELL" == "/usr/bin/bash" || "$SHELL" == "/bin/bash" ]];then
  if [[ "$UID" -eq 0 ]];then
    PS1="\[\033]0;\u@\h:\w\007\][\[\033[31;1m\]\u\[\033[0m\]@\h \W]# "
  else
    PS1="\[\033]0;\u@\h:\w\007\][\u@\h \W]\$ "
  fi
fi
fi

Change logging setup:

# Linux-ify the AIX logging setup and enable automagic rotation
# Everything but mail and auth to messages
*.info;mail.none;auth.none      /var/log/messages       rotate size 10m files 10 compress
# Auth to secure
auth.debug                      /var/log/secure         rotate  size 10m files 10 compress
# Mail to maillog
mail.debug                      /var/log/maillog        rotate size 10m files 10 compress
# Emergency messages to all users
*.emerg         *
*.info;mail.none                @NETWORK_LOG_SERVER

Remove "Message forwarded from hostname:" from remote logging output:

chssys -s syslogd -a "-n" ; stopsrc -s syslogd ; startsrc -s syslogd

Run aixpert to enable a much higher level of security:
aixpert -l high