Additions and Changes Made to the Netscape Messaging Access SDK =============================================================== Additions --------- base64.h 5/19/00 - Added base64 functions 11/10/00 - Added parameter to rfc822_binary() to control whether we want to have CRLF sequences added to the encoded string. md5.h 10/20/99 - Added basic MD5 algorithm. base64.c 5/19/00 - Added abse64 functions 11/10/00 - Added parameter to rfc822_binary() to control whether we want to have CRLF sequences added to the encoded string. md5.c 10/20/99 - Added basic MD5 algorithm. Changes ------- imap4.h 6/5/99 - Added a new exportable function imap4_clearResponses() to clear any pending responses. 10/20/99 - Added functions for authentication implementation. 10/26/00 - Added imap4_isConnected(). 11/14/00 - Added imap4_processPLAIN(). 3/2/01 - Added imap4_startTLS, imap4_getSocket. 5/16/01 - Added imap4_stopProcessResponses(). 10/16/01 - imap4_auth() and smtp_auth() now take a username and password which are used as initial client data for the AUTHENTICATE command. Also removed all traces of the CDSI macro, since it is no longer useful. imapdata.h 3/2/01 - Added support for STARTTLS command. 5/16/01 - Added field to imap4Client structure. mime.h 3/10/99 - Added a parameter to mime_messagePart_setMessage() to control whether to copy memory or not. 6/21/99 - Added exportable functions that enable the caller to decode base64 and quoted-printable buffers. 8/12/99 - Added mime_decodeModifiedBase64Buffer() and mime_encodeModifiedBase64Buffer() functions. 8/26/99 - Added a parameter to mime_decodeModifiedBase64(). 5/11/00 - Added mime_messagePart_getBasicPart() to retrieve the attachment from unknown message content types. 11/30/00 - Added getDefaultFileExtFromContentType(). 12/9/00 - getDefaultFileExtFromContentType() now takes a mime_content_type instead of a string. 12/18/00 - Added MIME_ENCODING_UU to mime_encoding_type enumeration. 9/7/01 - Added mime_multiPart_getBasicPart() and parameter to mime_message_putByteStream(). 9/12/01 - Added mime_message_associateByteStream. mime_internal.h 4/29/99 - Added a data file name field to the mime_basicPart_internal structure so that we can delete the file associated with the pTheDataStream field if need be. 5/11/00 - Added field to mime_messagePart_internal structure to handle unknown message content subtypes. 9/7/01 - Added mime_basicPart * field to mime_multiPart_internal_t. 9/12/01 - Added pInputStream field to mime_message_internal. mimeparser.h 11/29/00 - Added stristr(). 9/7/01 - Added boolean field to mimeParser_t. mimeparser_internal.h 4/29/99 - Replaced the message data vector field with a message data output stream in the mimeParser structure. nsmail.h 5/29/00 - Added logon error 3/2/01 - Added NSMAIL_ERR_TLS_INIT. 4/21/02 - Added NSMAIL_ERR_TLS_BADVERSION to detect when the SSL version needed by the server doesn't match what we are trying to connect with. 5/3/02 - Renamed NSMAIL_ERR_TLS_BADVERSION to NSMAIL_ERR_TLS_BADMETHOD to be more consistent with the nomenclature used elsewhere. smtp.h 5/19/00 - Added the auth, LOGIN and CRAMMD5 11/1/00 - Added initial support for AUTH PLAIN mechanism. Added smtp_processPLAIN(). 3/2/01 - Added smtp_startTLS, smtp_getSocket. 10/16/01 - imap4_auth() and smtp_auth() now take a username and password which are used as initial client data for the AUTHENTICATE command. Also removed all traces of the CDSI macro, since it is no longer useful. smtppriv.h 5/19/00 - Added the Auth Response 5/29/00 - ADDED AUTH END 6/12/00 - removed AuthEnd since it had no more use 3/2/01 - Added support for STARTTLS command. util.h 8/12/99 - Added decodeModifiedBase64() function. 8/26/99 - Added decodeQPHeader() to handle RFC2047 style quoted printable. 3/19/01 - Changed definition of BASE64_NOFIT and BASE64_SOMELEFTto make them unique error codes. 10/16/01 - Added generatePlainClientData(). imap4.c 3/12/99 - Commented out code that was causing the app to crash when it encountered very long header fields. The individual header fields do not need to be parsed as of right now. 3/29/99 - Increased the size of memory buffers by 1 to take the NULL character into consideration when doing reads. 4/1/99 - Added code to parseMailboxInfo() so that the SDK no longer chokes on folder names that are string literals. 4/6/99 - Did more cleanup of memory buffer sizes so that now both malloc() and memset() calls are fixed. 4/20/99 - Added getExtraLiteral() function that retrieves the next line from the server if we come across the start of a string literal. Added code to handle parsing string literals when parsing the BODYSTRUCTURE response as well as all non-FETCH responses. 5/6/99 - Added code to parseFetch() to handle fetching several body parts at the same time. Added a new exportable function imap4_clearResponses(). 5/21/99 - parseMailboxInfo() can now handle parsing folder names that are literals. 7/8/99 - Updated resolveFetchDataItem() so that it BODY responses are handled the same as BODYSTRUCTURE responses. 7/15/99 - Added a check in parseStatus() to see if the folder name is actually a string literal. 8/17/99 - Fixed a parsing problem in parseAcl(). 8/23/99 - Updated ACL functions so that parse quoted strings, and string literals properly. 8/25/99 - Fixed code in parseMyRights() and parseListRights() to handle end of line parsing. 10/20/99 - Added functions for authentication implementation. 11/3/99 - Fixed parseStatus() to handle folder names with '(' characters in them. 2/4/00 - Added code to parseNoBad() to handle getting a '*' at the beginning of the response line. Added code to parseMailboxInfo() to handle getting a NIL hierarchy delimiter. 5/19/00 - Put the rfc functions in base64.c. Removed some none functional functions. 9/8/00 - Fixed matchBrackets() so that it now ignores parentheses that are inside of quotes. 10/23/00 - getExtraLiteral() now takes an extra parameter - the number of chars in the literal. We now get at least that many characters in getExtraLiteral(). This fixes a bug that was caused by literals that contained embedded CRLF sequences. Callers of getExtraLiteral() now need to calculate the number of chars in the literal as well. 10/26/00 - Added imap4_isConnected(). It checks to see if the socket is still valid. 10/27/00 - Improved imap4_isConnected() to recognize the case where the server has disconnected us because of inactivity. 11/10/00 - Added parameter to rfc822_binary() to control whether we want to have CRLF sequences added to the encoded string. 11/14/00 - Added imap4_processPLAIN(), which knows how to properly handle the challenge and response for the AUTHENTICATE PLAIN command. 11/24/00 - Added code to matchBrackets() to ignore literal strings when trying to find the correct right parentheses. This stops us from having forever looking for string literals that do not exist. 11/27/00 - Added code to resolveFetchDataItem() to better recognize the case where we are asking for header info, so that in_pimap4->bReadHeader is set to TRUE. Although we don't currently care about the value of this variable, we might at some point want to use it. 12/2/00 - Added code to make imap4_isConnected() more robust. We now check for a SOCKET_ERROR return value from select(). Combined code to deal with SOCKET_ERROR return values from select() and recv(). 3/2/01 - Added workaround for compiler optimization bug in imap4_processCRAMMD5(). Added imap4_startTLS, imap4_getSocket for STARTTLS support. 4/19/01 - Now send a blank authorize-id for PLAIN authentication. This is allowed by RFC2595 and this allows to login to Netscape 4.15 servers that do not like us sending both and authorize-id and authenticate- id. Also fixed a silly typo in smtp_startTLS(). 5/3/01 - Fixed bug in parseFetch(). When parsing a BODYSTRUCTURE response from the server, we were calculating the beginning of the response under the assumption that BODYSTRUCTURE would be in the first block of bytes read from the server. 5/10/01 - We now handle getting back an empty list of ACLs in parseAcl(). We used to return a parse error. 5/16/01 - Added field to imap4Client structure that keeps track of when we want to ignore responses from the server. We check for this boolean being true in imap4_processResponses() and parseFetch(). Added check in imap4_isConnected() to only bother checking the state of the socket if we are actually connected. If we aren't connected, we haven't initialized the socket properly. Added imap4_stopProcessResponses() which sets the field in the imap4Client structure. 7/18/01 - Added check to see if we want to ignore the server response to the loop in parseFetch() for parsing the message body. This dramatically improves the responsiveness of cancelling (or skipping) when downloading an email with a large attachment. 8/14/01 - Removed code in parseEndingLine() and parseBye() that was freeing memory that was never allocated. This code caused a crash. 10/12/01 - Added local function RemoveAllPendingCommands() to take care of freeing items from the pending command list. It is now called in imap4_free(), imap4_disconnect(), parseBye(). The pCommand field of the pending command list needs to be freed whenever an element is removed from the list. As a result we need to allocate memory for that field in imap4_connect(). Fixed memory leak in parseMyrights() and parseListrights(). 10/16/01 - imap4_auth() now takes a username and password which are used as initial client data for the AUTHENTICATE command. Also removed all traces of the CDSI macro, since it is no longer useful. imap4_processPLAIN() now calls generatePlainClientData() to generate the data to send to the server. imap4_auth() now calls generatePlainClientData() as well if we are trying to authenticate using plain and both username and password were passed in. We are allowed to send initial client data when using PLAIN without waiting for an intermediate server response. 12/6/01 - parseMailboxInfo() now always passes an unquoted folder path to the callback function. Now the callback function doesn't need to care about the low-level IMAP details of how folder names need to be sent across the wire. Also, we now remove quoted characters (e.g. '\\') from the strings returned. This lets us handle the Domino IMAP4 server in a better way - it has "\\" as its hierarchy delimiter. mime.c 4/6/99 - Previous changes made to this file seem to have been rolledback. Did cleanup of memory buffer sizes so that now both malloc() and memset() calls are fixed. Added an extra parameter to mime_messagePart_setMessage() to handle shallow copying. 4/7/99 - Added a fix to prevent a buffer overrun when dealing with very long header field values. 4/29/99 - Made changes to mime_decodeBase64() to better handle the last few bytes of a stream we're trying to decode. 6/21/99 - Added exportable functions that enable the caller to decode base64 and quoted-printable buffers. mime_decodeQP() now calls decodeQP(). 8/12/99 - Added mime_decodeModifiedBase64Buffer(), mime_encodeModifiedBase64Buffer() and mime_bufEncodeModifiedB64() functions. Fixed code to prevent warnings. 8/26/99 - Added a parameter to mime_decodeModifiedBase64(). Now all of the encoding and decoding functions return an nsResult. 8/30/99 - Fixed encodeHeaderString(). 9/8/99 - Added code to getFileMIMEType() to handle Excel files. 9/9/99 - Update getFileMIMEType() to handle RTF and to return the correct type for excel documents. 9/10/99 - In decodeQP(), increated the size of achWriteBuffer by 1 to prevent croak!!! 11/3/99 - Added a fix to mime_decodeQP() to handle the case where a read buffer doesn't have all of the data required in its buffer (it needs bytes that will only be in the next read buffer). 3/7/00 - Modified mime_decodeQP() so that it only copies CRLF sequences, not arbitrary CR of LF characters. Modified getFileMIMEType() so that the content type parameter is no longer hard-coded to us-ascii, and the encoding used is quoted printable instead of 7bit (i.e. no encoding). This should allow attachments with foreign characters to be encoded properly. 3/23/00 - mime_basicPart_setDataStream() now handles zero length streams better when we only to save a reference to the stream. 5/11/00 - Now return MIME_ERR_NOT_FOUND in mime_messagePart_getMessage() if there's no embedded message. Added mime_messagePart_getBasicPart() for unknown message content subtypes. 11/14/00 - Handle ics extension by setting subtype to calendar. 11/30/00 - Added getDefaultFileExtFromContentType(), which gets the default extension for a MIME content type (e.g. text/plain -> "txt") 12/9/00 - getDefaultFileExtFromContentType() now takes a mime_content_type instead of a string. 12/15/00 - Fixed a bug in mime_decodeQP(). We weren't handling the case where there were stray bytes at the end of a read buffer properly. We now no longer lose those stray bytes. 12/18/00 - Fixed warning in getDefaultFileExtFromContentType(). 3/16/01 - Now ignore BASE64_SOMELEFT error code coming out of decodeBase64() in mime_decodeBase64Buffer(), so that we don't return an error when there are stray bytes at the end of a buffer that we could ignore. 6/5/01 - Added support for MS Powerpoint and MS Access content types to getFileMIMEType() and getDefaultFileExtFromContentType(). 9/7/01 - Added code to mime_multiPart_putByteStream() to ensure that we don't output the boundary parameter in the Content-Type field twice. Added mime_multiPart_getBasicPart(). Added parameter to mime_message_putByteStream() so that we can output only top-level headers. 9/11/01 - Removed stray fprintf() call that was being output when running exmigrate. 9/12/01 - Added mime_message_associateByteStream, which associates an already built byte stream with a mime_message. This stream is now used if present when mime_message_putByteStream() is called. 10/12/01 - In mime_basicPart_putByteStream(), we need to duplicate the memory before passing it into buf_instream_create() so that when we free it we don't free the szMessageBody field of mime_basicPart. Also fixed a memory leak to do with the buf_instream. 10/12/01 - Added code to mime_basicPart_deleteData() to properly free the pTheDataStream's memory. This function is currently not called by anyone. 12/7/01 - Modified mime_decodeQP() to handle .vcf file 4/28/02 - Added code to algorithms that read from a stream into a buffer to ensure that we don't try to read data into the buffer beyond the bounds of the buffer's memory. We do this by making sure that if we right at the end of the stream, that we only try to read however many bytes are left to be read - not MIME_BUFSIZE bytes. Otherwise we could crash. mime_internal.c 3/12/99 - mime_header_apend() now handles values that are more than 512 bytes long. 4/13/99 - Added code to mime_basicPart_internal_clone() to handle cloning a basic part with a data stream set. It now reads from the stream and fills a buffer. This needs to be improved in the case of large files. 4/23/99 - In mime_header_new() when szName == NULL. If szValue == NULL however assume empty string. 4/29/99 - Added code to initialize and free the pDataFileName field of the mime_basicPart_internal structure. 5/11/99 - Added a fix to mime_header_new() to prevent a crash. Now mime_header_add() calls mime_header_new(). 5/11/00 - Added handling for extra field in mime_messagePart_internal_t structure. 12/13/00 - Now return MIME_ERR_MAX_NESTED_PARTS_REACHED in mime_multiPart_addPart_clonable() when we try to add more parts that is allowed to a multipart. This stops a crash when downloading a message with "too many" multipart parts. 12/18/00 - Added code to recognize "x-uuencode" to mime_translateMimeEncodingType(). 1/20/01 - We do not want to add header fields whose name or value is empty in mime_header_add(). This stops a crash when cloning the extra_headers field. 9/7/01 - Added mime_basicPart * field to mime_multiPart_internal_t. So added code to mime_multiPart_internal_new(), mime_multiPart_internal_free(), mime_multiPart_internal_clone() to manage it. 9/12/01 - Added pInputStream field to mime_message_internal, and added code to mime_message_internal_new(), mime_message_internal_free() and mime_message_internal_clone() to support it. 10/12/01 - Now free the memory allocated for the pTheDataStream field in mime_basicPart_internal_free(). This could cause problems if the caller tries to free the same stream again. 4/28/02 - In mime_basicPart_internal_clone(), added check to optimize how we deal with very large text parts. If the size of the stream is above a certain threshold, we create a stream - otherwise we fall back on the old code of just allocating a buffer. Added code to algorithms that read from a stream into a buffer to ensure that we don't try to read data into the buffer beyond the bounds of the buffer's memory. We do this by making sure that if we right at the end of the stream, that we only try to read however many bytes are left to be read - not MIME_BUFSIZE bytes. Otherwise we could crash. Added code to all internal clone functions to stop allocated memory being overwritten and thereby creating a memory leak. mimeparser.c 1/14/99 - Fixed a bug in decodeDataBuffer() that was causing problems with very long lines in message bodies. 3/10/99 - Added a fix to mimeParser_parseMimeMessage() that stops it from choking on messages with lines over 1024 characters long. 3/11/99 - Fixed code that was assigning static text strings that were later being changed. Now use strdup() instead. 3/24/99 - Now zero out a buffer that had garbage at the end of it sometimes in mimeParser_parseMimeMessage(). 3/25/99 - Now only zero out of the buffer in mimeParser_parseMimeMessage() at the beginning of the call, not in the middle of the loop. 3/29/99 - Increased the size of memory buffers by 1 to take the NULL character into consideration when doing reads. 4/6/99 - Did cleanup of memory buffer sizes so that now both malloc() and memset() calls are fixed. 4/7/99 - Added a fix to prevent a buffer overrun when dealing with very long header field values in mimeParser_parseLine(). Added code to deal with long lines in mimeParser_parseMimeMessage(). 4/23/99 - Fixed a memory leak in addHeader(). 4/29/99 - Added code to store the encoded and decoded message data bytes in streams as opposed to memory. This dramatically speeds up parsing long attachments. Made a correction to a fix dealing with mime headers. Added a fix to properly calculate the offset in mimeParser_parseMimeMessage(). Fixed more memory leaks. 5/20/99 - Added code to fix a parse error that caused problems with parsing several embedded message attachments at once. 6/23/99 - Now add the proper CRLF to folded mime header field values in mimeParse_parseLine(). 8/17/99 - Fixed a bug that was causing the preamble to show up as part of the message body for deeply nested embedded messages. 10/18/99 - Commented out some code that resulted in a '\n\r\n' sequence being appended to every line that is parsed in a quoted printable message body. Now append '\r\n' to each line instead. 4/21/00 - Fixed a bug in mimeParser_parseLine() that was not outputting lines of message data that were empty (i.e. only had \r\n). 5/4/00 - Added code to mimeDynamicParser_free() and decodeDataBuffer() to better clean up the pMessageDataOutStream field of the mimeParser_t structure. 5/11/00 - Added handling for unknown message content subtypes, especially message/delivery-status and message/disposition-notification. 5/12/00 - Fixed mimeParser_parseForBoundary() so that it handles long boundaries properly. Now as many bytes as possible are stored, up to 70 (as per RFC 2046). 10/20/00 - Commented out some lines that were causing emails not to show up if the content- headers were ordered badly (i.e. Content-Transfer-Encoding before Content-Type). 11/29/00 - Added stristr(), which we now call in mimeParser_parseForBoundary() since we need to be able handle when "boundary" is not always in lowercase. stristr() does a case-insensitive substring search. 12/7/00 - Added code to properly set the content type in nNewMessageStructure(). It was getting lost. 12/18/00 - Fixed warnings in mimeParser_setData() and stristr(). 12/19/00 - Added code to mimeParser_parseLine() to add MIME header fields to the parent message part as well as the in usual message part. This fixes a bug where we were not able to display "Content-" header fields when trying to show all Internet headers. 1/15/01 - Commented out code in nNewMessageStructure() so that we do not choke on message/partial content type. Put fix in call to addHeader() in mimeParser_parseLine() so that we do not add headers to a queue in all cases. Now return MIME_CONTENT_APPLICATION as default content type when we come accross an unknown unknown media type in mimeParser_nGetContentType(). 1/20/01 - Rearranged code again in mimeParser_parseLine() to do with adding Content-xxx header fields to the list as well. Conditionals were still missing around the call to addHeader() that could potentially change the logic path, which we do not want. Now there is just a simple added call to addHeader() when we're dealing with "Content-xxx" headers. 2/20/01 - Fixed a bug in mimeParser_parseMimeMessage() in the case where we did not have enough bytes at the end of input buffer to determine whether we were dealing with a wrapped line. 3/26/01 - Fixed buffer overrun bug in mimeParser_parseForBoundary() that would happen when the boundary string in an email was longer than the RFC allows. We would write a NULL terminating character in memory not associated with a buffer. Fixed it by allocating a new buffer to copy the long boundary string into in the case when the buffer would be too long for our static character array. 9/7/01 - Added boolean field to mimeParser_t to control whether we should be treating the data that is being parsed as opaque. It's set to TRUE when parsing multipart/signed parts. When we parsing opaque data, we cache the data in a basic part that was added to multipart structure. Added code to correctly format that stream for multipart/signed parts - we need MIME header fields at the beginning of the stream and the trailing boundary at the end of it. Fixed code in mimeParser_parseLine() to put CRLF at the end of the preamble lines as required by the RFC. Also added code to mimeParser_parseMimeMessage() to ensure we put CRLF when folding header fields. mimeParser_checkForLineType() now ignores start boundaries when we are parsing opaque data - it only cares about trailing boundaries. 1/29/02 - Improved the logic of the MIME parser so that we detect when we the header fields are finished (i.e. when we come across one blank line in the message). This fixes the case where we were treating all of the lines of a message as if they were header fields (because they contain ':' characters in them). 4/14/02 - Fixed stupid mistake when building the opaque file for a multipart/signed part. We used to add the parameters for the Content-Type field to the MIME-Version instead. It was a simple matter of putting the two fields in the wrong order. This fixes Bug #1211. 4/28/02 - Added code to prevent small memory leak in nNewMessageStructure(). nssocket.c 10/27/00 - We only call WSAStartup() and WSACleanup() if DISABLE_NSSOCK_INIT is not defined. This is because we now call these functions when cstmsg32.dll is loaded and unloaded. smtp.c 5/19/00 - Added the Auth Response 5/19/00 - Added the Auth with LOGIN mechanism (Cram-Md5 not tested) 5/19/00 - Auth and login working 6/12/00 - Changed how the processResponse from the authentication is made. Now, an Auth Element is put in the pending list at the end of smtp_auth. At the begining of the smtp_processLogin, we check the pending list for Auth and return an error if it isn't. But the auth element is not removed from the list. It will be used in the smtp_processResponses call afterwards. AUTHEND is not put in the list and may be eliminated. 10/31/00 - Cleaned up smtp_processCRAMMD5() so that it actually works, and to properly check for error conditions. Replaced // comments with /* */ comments, since they are not normally recognized by C compilers. 11/1/00 - Added initial support for AUTH PLAIN mechanism. Added smtp_processPLAIN(). 11/10/00 - Added parameter to rfc822_binary() to control whether we want to have CRLF sequences added to the encoded string. Since IO_send() appends a CRLF sequence itself, we don't want to send two of them sequentially. This was causing a problem with AUTH LOGIN on the mirapoint SMTP server. Also added code after rfc822_binary() calls to make sure that we do not send any non-base64 characters to the server. 11/25/00 - Fixed busted smtp_processPLAIN() so that it now works. 3/2/01 - Added workaround for compiler optimization bug in smtp_processCRAMMD5(). Added smtp_startTLS, smtp_getSocket for STARTTLS support. 4/19/01 - Now send a blank authorize-id for PLAIN authentication. This is allowed by RFC2595 and this allows to login to Netscape 4.15 servers that do not like us sending both and authorize-id and authenticate- id. Also fixed a silly typo in smtp_startTLS(). 10/16/01 - smtp_auth() now calls generatePlainClientData() if we are trying to authenticate using plain and both username and password were passed in. We are allowed to send initial client data when using PLAIN without waiting for an intermediate server response. smtp_processPLAIN() now calls generatePlainClientData() to generate the data to send to the server. 1/22/02 - We can now properly handle multi-line responses to all commands in smtp_processResponses(). This fixes a problem where we were not processing server responses properly when the server gave us a multiline welcome message. util.c 4/29/99 - Added an extra check to allow callers to specify that they want to handle any bits still in the queue in decodeBase64(). 6/21/99 - Fixed the implementation of decodeHeader() and decodeQP(). 8/12/99 - Added decodeModifiedBase64(), decodeBase64WithTable() functions. Fixed code to prevent warnings. 8/26/99 - Added decodeQPHeader() to handle RFC2047 style quoted printable. 8/30/99 - Initialize ch = 1 in decodeQPxx functions 3/7/00 - Modified decodeQP() so that it only copies CRLF sequences, not arbitrary CR or LF characters. 2/9/01 - Modified decodeHeader() to better conform to RFC822. We now ignore whitespace characters between encoded words. 2/10/01 - Stopped a crash in decodeHeader() when we don't find the end of an encoded word. This crash would only manifest itself when downloading mail, and it would be caught by our exception handling. 6/8/01 - Modified the behavior of szLTrim() and szRTrim() so that if the string they are passed is composed completely of whitespace characters, that they both leave one whitespace character in the resulting string. This works around a bug in mimeParser_parseMimeMessage() where folded headers lines were not being handled properly if they contained only a space. 10/16/01 - Added generatePlainClientData() which takes a username and password and generates the data string that needs to be sent to the server when authenticating using the PLAIN mechanism. This code was extracted from the IMAP4 and SMTP functionality and put here since it is called now from many spots. 12/5/01 - Added code to decodeqp() and decodeqpheader() to ensure that we don't read beyond the end of the input buffer. This was causing a very hard to reproduce crash. Also added an assert() at the end of each function to ensure that we haven't gone beyond the input buffer.