Thanks for your input Jason. It helped me look at other aspects rather than worrying about ASE itself. I checked MD5 sum of all the main files and they were ok. I never got the tar file, rather a CD with directories containing install software. The issue got resolved and the reason as I understand it was - lack of /tmp space. The 10Gb partition was nearly full (98%) after my first installation attempt. I cleaned it up and removed a lot of stuff including some remnants of first install attempt and also removed all directories in $SYBASE directory from the same attempt to do a fresh install.
A bit surprised by the way AIX hogs /tmp space. It is still 85% full after installation is over.