FreeBSD vs WIN2K IPSEC road warrior HOWTO

Anton. R. Ivanov, IP Access LTD,
© 22.02.2002 ai1@ipaccess.com


 $Id: FreeBSD-WIN2K-IPSEC-HOWTO.html,v 1.9 2002/02/25 10:08:06 ai1 Exp ai1 $


1. Introduction and Disclaimers

After some experimenting with cross-compatibility between OpenBSD, FreeBSD, NetBSD, Windows2000 and Linux FreeSwan I have found out Windows IPSEC+PPTP RAS to be most easily implemented on FreeBSD. NetBSD should be similar. Summary: I have found FreeBSD on a common desktop class machine to be sufficient for the needs of a small enterprise. It can support up to 256 clients, more than 4 MBit 3des and more than 10MB des throughput on a 700MHz PIII. If you need more than this you most likely will have to go for a specialized access point from Cisco, Nortel or other vendors and/or specialized client software that supports algorithms more efficient than 3des.

Please keep in mind that all usual disclaimers apply:

  • This worked for me, your mileage may vary.
  • My employer denies any responsibility for any damages or losses caused by the use or misuse of this information. I deny any reponsibility as well.
  • All trademarks are property of their respective owners.
  • This material is based on the excellent OpenSSL cookbook, OpenSSL documentation and various documents from Microsoft , Verisign, the KAME project and others. All information used in this HOWTO is either freely available on the Internet or can be derived from elementary interoperability testing.
  • I will be glad to accept any corrections and amendments to this document and maintain it.
  • You are allowed to redistribute this document free of charge, modify it as you see fit and include in other documents and products as long as the original copyright notice is retained and/or due credit is given.
  • 2. Certificate Authority.

    If you already have one - go to the next section. Otherwise you need to create a certificate root.

    2.1. Create the necessary directory structure

    Choose your own directory location. Depending on the number of certificates to deal with and local policy - either /usr/local/etc/openssl or /var/openssl. Create the necessary subdirectories and files:

    2.2. Prepare an openssl.conf file

    The only reason for doing this is to avoid having to enter your company name, location, etc every time you run openssl. Here is a sample config based on the SSL Cookbook. Note that the v3 x509 extensions have been commented out. I have had some trouble with them and they are not used by any of the parties concerned anyway.

    2.3. Create the CA certificate and make a copy of it in PKCS#12 format so Windows can read it.

    As a result you will have two PEM format files for your Certificate Root and a PKCS#12 file which you can import into a windows system. Keep in mind that if you do not use the -nokeys switch, the resulting PKCS#12 file will contain your CA private key as well. At least with OpenSSL 0.9.6c under debian-woody this option does not work correctly and the keys are present in the PKCS#12 file (according to windows). If your private key has been kept for any reason it can be easily stripped by exporting it under windows and reimporting it back in.

    2.4. Create a cert request for the server, sign it and make a non-encrypted copy of it.

    Note that the openssl certificate request creates an encrypted private key. It has to be decrypted in order for racoon to be able to use it. That is the reason for the last openssl rsa line.

    2.5. Create a cert request for each windows machine, sign it and make windows readable PKCS#12 copy of it.

    Similar to the above: You need to repeat step 3.5 for every user and import the certificate into the windows system as described in 4.1.

    You will also need to copy the certificates to the FreeBSD VPN system and symlink them to their checksums (see section 3.2).

    3. Setting up a FreeBSD box.

    3.1. Rebuild your kernel with IPSec.

    You will need to enable the following in the config file: Also enable the firewall of your choice IPFIREWALL or IPFILTER. Keep in mind that some options like NAT may not be compatible with using IPSEC.

    3.2. Build and install KAME racoon from the ports collection.

    It is under /usr/ports/security/racoon. In most cases a simple make install should be enough. Best of all put the config under its own directory like /usr/local/etc/racoon/. The config is quite simple: Racoon searches for certificates based on their checksum. So, Client certificate under the cert directory will need to be symlinked to its magic hash name.

    3.3. Build and install the PoPToP PPTP server.

    It is under /usr/ports/net/poptop. FreeBSD by default builds it with userland ppp which does not support RC4 encryption. Considering that encryption is delegated to IPSec this is not a problem. Alternatively there is another VPN daemon and kernel ppp both of which support RC4 but I have not tested them. At least the kernel ppp with the RC4 patches is reported to be unstable. It may also be illegal to use it in some countties.

    After building it you need to add the following section to the bsd ppp options file.

    Update the IPs in the pptpd.conf file as well. I have not had the time to try what happens if they differ. If they are the same it works.

    3.4. Create a traffic policy

    Traffic policy on the BSD side is fairly simple. All GRE packets and PPTP control channel traffic is set to require IPSEC.

    4. Windows 2000 Setup

    You will need admin privileges for every client. There are also some prerequisites. Windows must be patched to SP2 and have the high encryption pack installed. The high encryption pack is available from . You will also need working network browsing. The easiest way to achieve this is to enable WINS on the PDC (or its samba equivalent) and ask PPP to supply the clients with the WINS server address.

    4.1. Certificate Setup

    Note, that this is the part where it is easiest of all to make a mistake. From the start menu run "mmc". From the console menu chose add-remove snap-in. Add the certificate and the IPSEC snap-ins. When asked which certificates do you want to manage chose local computer. Note that windows has per-user certificates as well as per-computer ones, but the per-user ones are not useable for IPSEC.

    Under the Certificates, right button click on Trusted Root Certification authorities. From all actions choose import . Import your certificate authority PKCS#12 certificate here. Check if it is displayed.

    Under the Certificates, right button click on Personal. From all actions choose import. Import the user PKCS#12 certificate here. Check if it is correctly displayed and the root authority is known.

    Under IP Security policies choose a policy which you are not using. Modify it as follows:

    If your personal certificate is correctly signed it will than be used for isakmp key negotiation. If you are having trouble, run racoon in debug mode on the BSD box. It will display the whole certificate in the debug output.

    4.2. PPTP setup.

    Create a new VPN connection. Make sure that under properties, in the security section the require data encryption field is left blank. You do not need it. This is the RC4 encryption for the PPTP which will be unnecessary overhead if IPSEC is protecting the connection.

    DEBUGGING and CAVEATS

    If you run into trouble your best chance is to use racoon in non-daemon mode with debugging enabled. I found most errors to be self-explanatory except one: This happens when the certificate used for the BSD server has an encrypted private key. You will need to run openssl rsa in order to decrypt it.