.NET Security Workshop
Home About Workshops Articles Writing Talks Books Contact

15. Data Protection API

Imagine this scenario: your code accesses a remote resource using a password. You store the password in the configuration file. However, this means that anyone with access to the file system will have access to the password. So you encrypt the password and store the cyphertext in the configuration file. Now when your process runs it reads the cyphertext and decrypts it. This raises a problem: where do you store the password to decrypt the cyphertext? You could store it in the executable file, but anyone with access to the file system will be able to read it. So you could encrypt that password but it now means you'll have to store that password somewhere...

The solution is to get the system to perform the encryption for you with a password that it maintains. This is done using the Data protection API.

This page is specific to .NET version 3.0/2.0.

15.1 Protected Data

The Data Protection API (DPAPI) is provided by the Local Security Authority (LSA) and the DPAPI calls are actually performed by a thread running in the LSA process so that the code is executed under the LSA security context. When DPAPI protects data it will return a blob that contains the encrypted data and information about how the data was encrypted. Encryption is performed with 3DES in CBC mode using a session key. This session key is never stored by DPAPI, instead it is generated at encryption time and information about how it was generated is put in the output blob so that DPAPI can generate the session key at decryption time. The central part of this session key generation is the user's password.

Since the session key is derived from the user's password it means that all processes running on the machine under the user's account may be able to generate the session key. To protect against this DPAPI can use two pieces of extra data: a secret blob called the secondary entropy that will be known by a specific process and DPAPI can also prompt the user to provide a password. The .NET classes that wrap DPAPI do allow you to provide entropy, but it does not allow you to prompt the user for a password.

DAPI will use a random 512 byte MasterKey to generate the session key. The MasterKey is stored in the user's profile on the hard disk in an encrypted form (using 3DES and a password derived from the user's password), but it is only valid for three months. When the MasterKey expires a new MasterKey is generated, although this new key will be used in furture encryption actions the old MasterKey will still exist on the hard disk. The password used to encrypt the MasterKey is created using PBKDF2 (the PKCS#5 Password Based Key Derivation algorithm). DPAPI also maintains a credential history file. When the user changes their password the old password is added to the credential history file and then the entire file is encrypted with a key derived from the new password. This way the old passwords can always be accessed and used to extract any MasterKey stored on the hard disk.

The session key is generated from the MasterKey, 16 bytes of salt, entropy and any additional password provided by the DPAPI prompt. It is this session key that is used with 3DES (in CBC mode) to protect the data. The output of the data protection will be a blob that contains the cyphertext, the salt, entropy and a GUID that identifies the MasterKey that was used. The session key is not stored anywhere, but the protected data blob has enough information to generate the session key when unprotecting the data. To protect the MasterKey from tampering a HMAC hash is created from the MasterKey, using SHA1 and the user's password and this HMAC is also encrypted with the PBKDF2 key and the cyphertext is stored in the user's profile.

The DPAPI gives you the option of using a machine key rather than a user key to protect data. This means that all users on the machine can unprotect data which only makes sense for data that is stored off machine.

.NET supplies the ProtectedData class as a wrapper around DPAPI. This class has two static members, Protect and Unprotect. Protect has three parameters: two byte arrays, one has the data to protect and the other has the entropy; and DataProtectionScope enumeration which can be CurrentUser or LocalMachine. Unprotect has the encrypted blob, the entropy and the scope enumeration.

To test this out create a file (protect.cs) and add the following code:

using System;
using System.IO;
using System.Security.Cryptography;

class App
{
   // Parameters: e|d infile outfile
   static void Main(string[] args)
   {
      if (args.Length < 3) return;

      bool bEncrypt = (args[0][0] == 'e');
      if (!File.Exists(args[1])) return;
      if (File.Exists(args[2]))
      {
         File.Delete(args[2]);
      }

      byte[] inData, outData;

      using (FileStream fsIn = File.OpenRead(args[1]))
      {
         inData = new byte[fsIn.Length];
         fsIn.Read(inData, 0, inData.Length);
      }

      if (bEncrypt)
         outData = ProtectedData.Protect(inData, null, DataProtectionScope.CurrentUser);
      else
         outData = ProtectedData.Unprotect(inData, null, DataProtectionScope.CurrentUser);

      using (FileStream fsOut = File.OpenWrite(args[2]))
      {
         fsOut.Write(outData, 0, outData.Length);
      }
   }
}

This code does not use entropy and so it passes a null pointer for the second parameter. The code merely opens the file given as the second parameter, reads the data into a byte array and then passes this array to either the Protect or Unprotect method. The encrypted data is then written to the file identified by the third parameter.

Compile this code (csc protect.cs) and then test it out with the following:

C:\Security\15.1> protect e protect.cs protect.dat
C:\Security\15.1> protect d protect.dat protect.txt
C:\Security\15.1> comp protect.cs protect.txt

The first line encrypts the source file, the second line decrypts the cyphertext and the final line compares the first file with the decrypted data. You should get a message Files compare OK and you will be prompted to compare other files, enter n to close the process.

15.2 Protecting Configuration Files

Earlier I mentioned the hypothetical (but common) situation of holding sensitive data in a configuration file. Clearly DPAPI provides a solution to do this. You could write a tool that reads the secret data and then encrypts it with DPAPI and writes this data to the configuration file. Then when the actual process runs it reads this appropriate section, decrypts the data with DPAPI and then uses the data. Such a scheme is possible, but before you start thinking about how to do this, stop. The .NET framework provides many new classes for configuration, and included in these classes is code to encrypt sections and write those sections to the configuration file. Once you have encrypted a section the configuration file classes will automatically decrypt the code when you read it: you do not need to explicitly decrypt the data.

Start by creating a file (conf.cs) and add the following code:

using System;
using System.Configuration;

class App
{
   static void Main()
   {
      Configuration config = ConfigurationManager.OpenExeConfiguration(
         ConfigurationUserLevel.None);
      AppSettingsSection appSettings = config.AppSettings;
      if (!appSettings.SectionInformation.IsProtected)
      {
         appSettings.SectionInformation.ProtectSection(
            "DataProtectionConfigurationProvider");
         appSettings.SectionInformation.ForceSave = true;
         config.Save();
      }
      else
      {
         foreach (KeyValueConfigurationElement element in appSettings.Settings)
         {
            Console.WriteLine("{0} = {1}", element.Key, element.Value);
         }
      }
   }
}

This code accesses the <appSettings> section and then tests to see if the section has been protected. If it has then the code simply reads every key-value pair in the section and prints out the data on the command line. If the section is not protected then the code protects the section using DPAPI and then saves the entire file. Compile this code (csc conf.cs).

To test this code you need to create a configuration file (conf.exe.config):

<configuration>
   <appSettings>
      <add key="data" value="secret data" />
   </appSettings>
</configuration>

Run the application. You'll find that nothing appears to happen until you type the configuration file to the command line (type conf.exe.config):

<configuration>
   <appSettings configProtectionProvider="DataProtectionConfigurationProvider">
      <EncryptedData>
         <CipherData>
            <CipherValue>
               AQAAANCMnd8BFdERjHoAwE/Cl+sBAAAAtw1dTrOaZEmRjoHbU5o7BgQAAAACAAAA
               AAADZgAAqAAAABAAAAAhL62oNkxHz042gZOLnAKoAAAAAASAAACgAAAAEAAAAISs
               t2KwFXF3T+Ug+y2ZESSYAAAAy9sV+eKT6uxTVrWW4qz2BPqoqMX1IecxQCDWeqG4
               89PV5Qsdxe2cwb45qD6IV2jWiaVjnEjl2QCNSvL4lzogfOFG+E+4lytCyDKfREbO
               FLc3xcsp6g6VqZmvAR/bKejYiaMo1sI/d+DTwIaNutDEGzu87TR3gkyLEOn3tMEj
               K6uoF+Otnn6djUWX8ExDLTzchTEor7hFa8cUAAAAiJ92RKXCTGkDIYhBFYnmgbwA
               pjM=
            </CipherValue>
         </CipherData>
      </EncryptedData>
   </appSettings>
</configuration>

The data that you had in the configuration file has been replaced with the base64 encoded DPAPI blob shown above. Now run the application again. This time the code will detect that the section is encrypted and it will decrypt the data automatically so that you'll see the following on the command line:

data = secret data

Notice that the configuration section has an attribute called configProtectionProvider and this indicates that the data is encrypted using DPAPI. To test this out, create a file (dpapi.cs) and add the following:

using System;
using System.Security.Cryptography;
using System.Xml;
using System.Text;

class App
{
   static void Main(string[] args)
   {
      if (args.Length == 0) return;
      XmlDocument doc = new XmlDocument();
      doc.Load(args[0]);
      XmlNodeList nodes = doc.SelectNodes(
         "configuration/appSettings/"
         + "EncryptedData/CipherData/CipherValue");
      byte[] buf = Convert.FromBase64String(nodes[0].InnerXml);
      byte[] unenc = ProtectedData.Unprotect(
         buf, null, DataProtectionScope.CurrentUser);
      Console.WriteLine(Encoding.Unicode.GetString(unenc));
   }
}

Compile this (csc dpapi.cs) and run it passing the name of the configuration file with the protected data:

C:\security\15.2> dpapi conf.exe.config
<appSettings>
   <add key="data" value="secret data" />
</appSettings>

As you can see, this is the data that was in the unprotected configuration file.

If you chose, you can use the RsaProtectedConfigurationProvider to encrypt the configuration section. The name is a little confusing because the actual encryption is performed with 3DES in CBC mode in a similar way to the DPAPI method shown above. The difference is that the data is encrypted with a random key that is itself encrypted using RSA key exchange and put in the output configuration section. A machine certificate is used in the RSA algorithm.

15.3 Protected Memory

The Windows API allows a process to inject a thread into another process running on the same machine; it also allows one process to access the memory of another process. Of course, both of these actions are subject to NT access security because they are privileged tasks. However, a rootkit or some other malware could be lurking in memory running under a privileged account, this is a situation that happens rarely, but even so, Microsoft decided to write code to mitigate such situations. Such malware could wait until your code accesses a remote resource using a password, or makes a purchase with your credit card, and then report this sensitive data to the malware's owner.

The ProtectedMemory class uses DPAPI to encrypt a blob of data in memory and overwrites that blob with the result. This is different to ProtectedData which will return a new buffer with the encrypted data leaving the original data untouched. The ProtectedMemory class has two methods, Protect and Unprotect. Protect has just two parameters, the first is a byte array with the data to encrypt and this array will have the encrypted result. The second parameter has the scope. Note that this method does not allow you to provide entropy.

If your code holds onto some sensitive data, for example a session key, then it makes sense to hold that data as encrypted data using the ProtectedMemory class. In this case you can decrypt the data just before you need to use it. Note, however, that it would be pointless to use protected data like this if you do not take steps to ensure that the unprotected data only exists for the short time when it is needed.

byte[] protected = GetProtectedData();
// ... other code
byte[] unprotected = ProtectedMemory(protected, DataProtectionScope.CurrentUser;
UseUnprotectedData(unprotected);
unprotected.Initialize();
unprotected = null;

This code initializes a buffer with protected data and then at some other point in its execution it unprotects (decrypts) the data so that it can use the sensitive data. After the information is used you must ensure that the unprotected data is removed from memory, and to do this I call Initialize which will call the default constructor on each member of the array, which in this case will initialize each member to a value of zero. After that I assign the array reference to null. This last line is not strictly necessary, but it does make the array unreachable so that when the GC runs it will reuse the memory used by the array.

If the sensitive data is a string then you should use SecureString. This class uses DPAPI but more importantly, it ensures that when the string is disposed the buffer containing the protected data is zeroed and the allocated memory is freed. This happens because the buffer is an internal class called SafeBSTRHandle and as the name suggests, the buffer is not allocated on the GC, instead, it is allocated as an OLE BSTR and when the buffer is released a call is made to the OLE SysFreeString.

I find it curious that a sensitive action like this must be performed using a COM type. This just shows that those people who insist that .NET will kill off COM are far off the mark: .NET depends on COM.

SecureString derives from CriticalFinalizerObject means that the finalizer code is marked as a constrained execution region. This means that the code in the finalizer can only call code that have a strong  reliability contract and cannot throw an out of band exception, for example, as a result of a fault during boxing or attempting to acquire a lock. A reliability contract makes guarantees about how reliable the method is (ie, is it likely to fail, or will it always succeed), and if the method can fail, the contract indicates whether the failure will corrupt the instance state, or the state of the application domain. The SecureString finalizer's contract indicates that it will not corrupt state if an exception occurs, however, some of the private methods, for example, the private method ProtectMemory method called when the instance is changed, indicate that they may corrupt the instance in an exception condition.

The String class is immutable, which means that you cannot change an instance; if you call one of the methods that appears to alter a string, for example Trim, the original is left unchanged and instead you are returned a new instance that has the alterations. Imagine what that would be like for a string with sensitive data, you could have several instances in memory that contains all, or some of the sensitive data. The SecureString class is mutable until you mark it as read-only (with a call to MakeReadOnly), this limits the number of copies that will be held in memory. However, note that you have few methods to alter the data in a SecureString: AppendChar, to add a character to the end of the string; InsertAt to insert a character at a specific position in the string; SetAt to change a particular character and RemoveAt to remove a specific character. The class has two public constructors: a default (parameterless) constructor and a non-CLS compliant constructor that takes a char* pointer and a character count. This means that you can either initialize an instance with a native Unicode string (a deep copy is made) or you create an empty string and have to initialize it character by character using AppendChar. Again, the reason for this is to prevent you from using buffers allocated on the GC which could have a long lifetime. An unmanaged string is assumed to either live on the stack or on an unmanaged heap, and in both cases the lifetime of that data is limited.

The example on page thirteen shows how you can initialize a SecureString using AppendChar; I have reproduced it here:

X509Certificate2 cert = null;
Console.Write("Password for certificate file: ");
using(SecureString ss = new SecureString())
{
   while (true)
   {
      ConsoleKeyInfo key = Console.ReadKey(true);
      if (key.Key == ConsoleKey.Enter || key.Key == ConsoleKey.Escape) break;
      Console.Write('*');
      ss.AppendChar(key.KeyChar);
   }
   Console.WriteLine();
   cert = new X509Certificate2(certFile, ss);
}

As you can see the SecureString is initialized a character at a time and each character is obtained from the console as the user types. ConsoleKeyInfo is a value type and so it lives on the managed stack, and so the characters used to initialize the SecureString have a limited lifetime. On each call to AppendChar the entire string is unprotected, then the character is appended to the end of the string, and then the string is protected.

SecureString implements IDisposable because the underlying storage of the data is in an unmanaged BSTR so you should call Dispose as soon as possible. It also makes sense to ensure that the sensitive data (even data that is protected) is only in memory for just as long as it is needed.

Currently there are only 6 classes in the framework that use SecureString. You have already seen one: X509Certificate2 makes the widest use of SecureString, the class is used for the password when importing and exporting certificates that have private keys. The CspParameters class also uses this class for passwords for keys - in this case for a smart card key. The other main use for SecureString is in the Process (and ProcessStart) classes and is used to indicate the password of the user account under which the process will run. The Marshal class allows you to convert a SecureString to an unmanaged string allocated using the GlobalAlloc method, the OLE VCoTaskMemAlloc API or as a BSTR; and to free such unmanaged strings. The remaining class that uses SecureString is SecurityUtilities but MSDN marks this as supports the .NET Framework infrastructure and is not intended to be used directly from your code.

I hope that you enjoy this tutorial and value the knowledge that you will gain from it. I am always pleased to hear from people who use this tutorial (contact me). If you find this tutorial useful then please also email your comments to mvpga@microsoft.com.

Errata

If you see an error on this page, please contact me and I will fix the problem.

Page Sixteen

This page is (c) 2007 Richard Grimes, all rights reserved