We had a pair of Checkpoint firewalls running FW-1 4.1 on Sun Ultra 10 boxes running Solaris 2.6.  Like everyone else we were faced with the fact that Checkpoint would be dropping support for FW-1 4.1, though they pushed back the date from Dec 31 2002 to June 30 2003.  Our 4.1 firewalls ran quite reliably, so there were no compelling technical reasons to perform the upgrade.

Ultra 10s are cheap on eBay - $700 for the latest and greatest, half that or less for older models.  QFEs are about $200.  For very little $ you can create your own lab with a pair of firewalls on your own time and fully test your upgrade before putting it into production.  Ultra 10s take 2 IDE hard drives - you can script a back up one drive to another for quick and dirty disaster recovery preparedness.  If your Checkpoint VAR botched the initial install of your firewalls like ours did leaving you with partitions too small to upgrade, you can also modify the partition sizes of your hard drives during your backup so you can successfully upgrade Solaris.  Here's the script and configuration file....

mirror_root        config

We use TrendMicro's Interscan Viruswall and RSA's ACE Server with our firewalls.  Fortunately we discovered during testing that the ACE server needed upgrading as well.  RSA tech support is excellent and seems to know more about Checkpoint than many of Checkpoint's own technicians.  Checkpoint has an article on their website happily declaring their support for ACE 5.x with NG.  What they fail to tell you was that NG doesn't support ACE 4.x at all since the APIs changed.

Upgrading the firewalls to NG FP3 was quite a project.  This cheatsheet is no substitute for RTFMing, but it should help.

1) If you have deployed SecuRemote, you'll need to have all your users upgrade to the latest FP3 SecuRemote before you upgrade your production firewalls.  In typical Checkpoint fashion you are faced with a Catch-22:

- SecuRemote FP2 and later create a site with a 4.1 firewall.
- VPN-1 FP2 and later won't authenticate an older SecuRemote.

As a workaround I upgraded SecuRemote on a PC that already had our sites defined, then distributed the upgraded userc.c to production SecuRemote clients via a login script.  I had my SecuRemote users uninstall SecuRemote 4.1 and install the newest SecuRemote because at the time an upgraded SecuRemote renamed itself SecureClient - this bug was later fixed in build 53515.

2) Apply 4.1 SP6 to the firewalls.  We had no ill effects; in fact it fixed one problem.  We use Nexland personal firewalls for remote access.  (Nexland was bought by Symantec in July 2003.) When I started using them 4 years ago, they were the only personal firewall to work with SecuRemote, and they are still the only one to support multiple SecuRemote sessions (though I seem to have lost this capability after the upgrade).  Multiple SecuRemote sessions from behind a Nexland didn't work very well until we upgraded from 4.1 SP3 to SP6.  We are now having good luck with the much cheaper Netgear MR814v2.  The original MR814 didn't work seamlessly with SecuRemote - most functions worked, but double clicking 'My Computer' on Windoze 2000 to see your mapped drives hung the computer.  This may have been an MTU size problem.

3) Upgrade the firewalls to Solaris 9.  Our upgrades wouldn't run until we created the directory /var/sadm/system/logs.

4) Apply the latest recommend patches for Solaris 9

5) Remove unnecessary packages.  I used Lance's Whitepaper's as a guide:

6) Here are the files I had to modify on the firewall after the upgrade:

/etc/inetd.conf – remove everything but ftp.  I leave ftp, but you could easily argue it should be removed as well.
/etc/nscd.conf - change positive ttl to zero – disable name service caching
/etc/sshd/sshd_config – allow root to login
touch /etc/notrouter
/etc/vfstab - enable logging on all partitions

7) Follow these tips in the FireWall-1 Performance Tuning Guide

/etc/rc2.d/S69inet – add these lines to the end of the file
# shut off access – fix security
ndd -set /dev/ip ip_forward_src_routed 0
ndd -set /dev/ip ip_forward_directed_broadcasts 0
ndd -set /dev/ip ip_respond_to_echo_broadcast 0
ndd -set /dev/ip ip_respond_to_timestamp_broadcast 0
ndd -set /dev/ip ip_send_redirects 0
ndd -set /dev/ip ip_ignore_redirect 1
# firewall fine tuning
ndd -set /dev/tcp tcp_slow_start_initial 2
ndd -set /dev/tcp tcp_conn_req_max_q 1024
ndd -set /dev/tcp tcp_conn_req_max_q0 4096
ndd -set /dev/tcp tcp_time_wait_interval 60000
ndd -set /dev/tcp tcp_xmit_hiwat 65535
ndd -set /dev/tcp tcp_recv_hiwat 65535

/etc/system - add these lines to the end of the file
set sq_max_size = 100
set autoup = 120
# both the above require >128M RAM
set tcp:tcp_conn_hash_size = 16384
set rlim_fd_max = 16384

8) Run the upgrade_checker from Checkpoint's website to look for any potential problems.  When I performed my upgrade the upgrade_checker only handled FP2, which I assumed was better than not checking anything at all:

# ./upgrade_checker -p /opt/CPfw1-41 -c 4.1 -t NG_FP2

9) Upgrade the firewall to NG FP3.  Looking at all the problems with the original NG, FP1, and FP2, and even FP3 hotfix 1, I was glad I waited so long.

10) Apply hotfix 2 for FP3.

11) Install the latest cpinfo from checkpoint's website.  cpinfo would not run on my machines until I created the files

/usr/sbin/crash
/usr/sbin/mdb

and chmoded both to 555. 

After all this work upgrading my existing firewall, I ended up throwing away all the rules and re-writing them to take advantage of the new simplified policy features.  I kept my objects.  My 'VAR' had provided one policy for each of our two firewalls, so this also allowed me to create a single policy which makes things much easier.

During testing I was unable to make SecuRemote create a site until selected the VPN domain setting 'All IP Addresses behind Gateway based on Topology information'.

What I learned the hard way after the firewalls were in production

1) Automatic ARP does not work unless you also use automatic NAT.  This detail was not documented anywhere, and I got a promise from Checkpoint that they would document it.  I got to learn 

debug ip arp

on the Cisco router.

2) I had to change policy values smtp_rfc821 and smtp_rfc822 both to false to avoid having the firewall strip incoming email fields that didn't comply with the RFCs.

3) I had to modify the Advanced -> SMTP properties on each firewall to allow inbound email over 1 meg in size.

4) My backup incoming mail server at my remote site was unable to connect to the Interscan Viruswall at my main site due to the automatic encryption domain.  I ended up changing my backup incoming email server to have it forward all email to my primary incoming email server, which then sent it to the Interscan Viruswall.

5) Install Hotfix accumulator 12.  We use Polycom Viewstations over VPN to provide 'free' videoconferencing between our sites. We had problems with them after the upgrade that were only rectified with HFA12.  Apparently you couldn't have an H.323 videoconference over 30 minutes before this hotfix.