Connections are dropped intermittently on rule 0 Error: "unknown established TCP packet" after recent upgrade to VPN-1/FireWall-1 4.1 SP3


Cause
New entries in the connection table are by default entered with a timeout of 60 seconds until the connection is established (i.e., a complete TCP handshake). Once the connection is established, the relevant timeout is the one defined in the Security Policy tab of the Properties Setup window. In VPN-1/FireWall-1 4.1 SP3, "once the connection is established" means when the first non-SYN packet arrives from the server after the TCP handshake. In previous versions of VPN-1/FireWall-1, "once the connection is established" meant when the TCP handshake was complete. That is when the client sent an "ACK" responding to the server's "SYN ACK". The new behavior introduced in VPN-1/FireWall-1 4.1 SP3 can cause connections to be dropped if there is a slow response, thus resulting in the above error message.


Fix
There are several workarounds. Choose which ever one best suits your company's security policy. Note that all security implications must be considered before implementing any change.

1. Increase the TCP start timeout from 60 to 3-4 minutes:

To change the TCP timeout for starting connections, perform the following steps.

1. Close the Policy Editor GUI.
2. Edit the objects.C file. Add
:tcpstarttimeout (###)
under the :props section (where ### is the timeout value in seconds). See SK294 below on how to edit objects.C.

3. Reinstall the Security Policy.

2. Force the VPN/FireWall Module to match non-SYN packets which do not belong to an established connection against the Rule Base:

Notice that you must have two rules:
client - server - Any - accept
server - client - Any - accept

To disable this security enhancement and allow "Non SYN" packets to be matched against the rule base follow these steps:

1. On the Management station, open the file
$FWDIR/lib/fwui_head.def

2. Find the line:

/*#define ALLOW_NON_SYN_RULEBASE_MATCH*/

3. Uncomment the line. Change it to

#define ALLOW_NON_SYN_RULEBASE_MATCH

4. Install the Policy.

**NOTE** You should consider all security implications when allowing this kind of traffic to be matched against the Security Policy

3. Restore the pre-Version 4.1 SP3 behavior, so the connection timeout will be measured from when the TCP handshake is complete. To apply the change on Solaris:

a. Edit FireWall-1 module /etc/system
b. At the bottom, under the line:
set test_module:debug = 0x13
Add:
set fw:fw_old_established_accept = 1
c. Save the file
d. The change will take affect only after a reboot

To apply the change on IPSO (Nokia):

1. Use the modzap utility to change the fw_old_established_accept variable in the VPN-1/FireWall-1 Module kernel with the following command:
./modzap fw_old_established_accept $FWDIR/boot/modules/fwmod.o 0x1
2. Reboot

Note: If you are running the VPN-1/FireWall-1 Module on NT, contact CheckPoint Support for a fix.

Warning: Observe the following basic rules for configuring the FWDIR/conf/objects.C file:

1. The objects.C file should be configured on the VPN-1/FireWall-1 Management Module only.

2. Before configuring the objects.C file:
- Close all VPN-1/FireWall-1 GUI clients
- Stop the Management Module using the fwstop command
- Delete the files objects.C.sav and objects.C.bak
3. Backup the original objects.C to another partition or folder.
4. Make the desired changes to the objects.C file by editing the $FWDIR/conf/objects.C file using a basic text editor, such as Notepad. Do not use a word processor.
5. Save the changes to the objects.C file
6. Restart the Management Module using the fwstart command and verify that the change you made is saved.
7. Install the Security Policy.
8. For properties that involve the security servers, VPN-1/FireWall-1 must also be restarted.

02/JUL/02 Jim Parker

< back