DocDeliver RELEASE NOTES: This document describes changes and bugfixes to the AXT DocDeliver outlet filter. Such changes may affect version upgrades, so if experiencing problems after a version upgrade, please consult this document and the major changes between the old and the new version. =============================================================== 2009-10-21 DocDeliver 3.1.7 =============================================================== + Improvement in handling of FTP timeouts. Jobs to servers that don't respond to commands within the timeout time are failed/retried. + The general handling of logins now consider communication problems as retryable whereas problems with credentials are still considered irrecoverable errors that require user interaction. + file/ftp deliveries that take too long time may be aborted using a combination of the maxDeliveryTimeSec, maxInactivityTimeSec and abortOnInactivity job properties. + file/FTP deliveries report activity more fine grained which means that jobs that progress slowly are still considered active whereas jobs that stop sending data for some reason are considered inactive and may be aborted after a specified inactivity time. + file/ftp deliveries that do not complete due to abortion or due to other errors during the upload may be retried by setting the property retryIncompleteTransfer The example here shows how a certain delivery can be given specific properties for the maximum total delivery time, the maximum time without reported activity on the line, if the job should be aborted on activity and if the job should even be retried if the delivery was aborted half the way through an upload. Note that the media configuration files state defaults and that you may define media configurations for specific media and servers as usual with different values for the properties as seen fit. + The DDS status display shows more detail on threads and delivery status =============================================================== 2009-09-02 DocDeliver 3.1.6 =============================================================== + Bugfix: Fixed a AXT keys parsing bug introduced in 3.1.4 when fixing character encoding for AXT keys. =============================================================== 2009-09-01 DocDeliver 3.1.5 =============================================================== + Bugfix: Fixed a ftp bug introduced in 3.1.3 which caused an error creating folder for FTP media, and other media inheriting functionality from the file system media. =============================================================== 2009-08-31 DocDeliver 3.1.4 =============================================================== + Added BCC option to mail media using the param: + Bugfix: AXT keys passed using the docdeliverOutletFilter caused an exception if they contained special characters like '[]@..' eventhough this is allowed in AXT. Fixed by passed axt keys as an embedded XML datastructure to the Docdeliver service. =============================================================== 2009-07-10 DocDeliver 3.1.3 =============================================================== + Significant performance improvements. The persistence layer has been rewritten resulting in a regular performance boost on big job collections. Improvements to job scanning performance, update with Log & Trace properties and more. + Deletion back-off ensure that periodic cleaning (deletion old jobs) consume limited resources while other activity is ongoing. The scheme can be configured using the properties dds.exist.deletion.maxstrech and dds.exist.deletion.backoff.ms (see docdeliver.properties). + Main settings file (docdeliver.xml) has been given a schema to support AGETOR Workbench configuration. ´ + Bugfix: when shutting down the service while periodic deletion were in progress the deletion failed on each job resulting in log flooding + Bugfix: Deletion status not shown correctly when simultanous deletions were in progress. Only the latter started was shown. + Job deletion history shown in status (previous n run) =============================================================== 2009-02-18 DocDeliver 3.1.2 =============================================================== + Changed docdeliver media configuration files overwrite during installation from true to false to avoid custom settings being overwritten during minor updates. =============================================================== 2008-10-27 DocDeliver 3.1.1 =============================================================== + Added support for SFTP stricthostkey checking by specifying the a hostfile which contains a list of known hosts. + Bugfix: Fixed a bug causing both succeeded and failed jobs to be deleted after a specified time period, even though the time periods should be independent. =============================================================== 2008-09-12 DocDeliver 3.1.0 =============================================================== + Added support for FTPS media (FTP over SSL) Note: FTPS usually requires the server to run in passive mode in these cases the activemode=false parameter must be set. + Added support for SFTP (file delivery to SFTP/SSH servers). SFTP (Secure File Transfer Protocol) is not based on the FTP protocol and should be thought of as a secure file system. =============================================================== 2008-06-03 DocDeliver 3.0.9 =============================================================== + Added support for email servers requiring authentication by adding the parameter: to the mail media. + Added support for extended email address in the from (sender) field. Ie. instead of jdo@acmecorp.com the email can be specified as: John Doe =============================================================== 2008-06-02 DocDeliver 3.0.8 =============================================================== + Fixed a bug that caused the configured DDS datafile extension to be ignored. =============================================================== 2008-04-10 DocDeliver 3.0.7 =============================================================== + Optimized job search functionality for AXT JOBS. + Fixed a bug where incomplete datastreams caused empty data files in the delivering folder. + Modified the default cleanup intervals, such that succeeded jobs now are deleted after 3 days and failed jobs after 30 days. This modification is made to increased overall performance on systems with heavy load. + changed deletion process to be nonblocking to allow continues processing of documents while deleting. + Improved docdeliver status response time and added jobs cleanup status section =============================================================== 2008-03-11 DocDeliver 3.0.6 =============================================================== + Optimized dds collection indexes in eXist Database for optimal performance. + Fixed a minor problem with resource not found warnings printed in the log file. =============================================================== 2008-02-21 DocDeliver 3.0.5 =============================================================== + This version requires ADK 3.0.4 since it relies on eXist version 1.2. + A number of important bug-fixes with respect to eXist access and synchronization. Under high load the access resulted in errors. + Potential deadlock removed that could occur under high load. + Significant improvement of parallelism in job execution - in particular it is possible to control the number of simultaneous AXT transformations that can be executed. =============================================================== 2008-01-17 DocDeliver 3.0.4 =============================================================== + Bugfix: Minor bugfix retryInterval filter attribute setting had to be lower case, the retryInterval as specified in the documentation did not work. + Bugfix: renameonupload option functionality had been removed unintended. + Bugfix: Encoding problem when fetching stored AXT documents + Bugfix: Internal docdeliver AXT calls with recursion could cause a deadlock if maxconnections was reached. + Bugfix: Synchronized jobs could cause hanging connections if job was delivered by scheduler, before docdeliver was notified to deliver the job. This could in rare cases cause for instance the ftpinlet to wait for job that had been delivered and thus ban the file. =============================================================== 2007-10-30 DocDeliver 3.0.3 =============================================================== + Added optional configuration of generated retry checkpoints using the global property docdeliver.retry.checkpointtext + Automatic retries now no longer results is a failed trace if it resolves the error automatically. + AXT jobs are not counted as active jobs in the DDS queues to avoid potential deadlock situations. + Variable substitution now also performed on inlined input =============================================================== 2007-09-28 DocDeliver 3.0.2 =============================================================== + Bugfix: Encoding problem fixed when editing and re-saving job-info from redelivery GUI. =============================================================== 2007-09-18 DocDeliver 3.0.1 =============================================================== + Bugfix: Enriched values was not stored/loaded by internal job XML handler. + A manual redeliver will now notify the Log and Trace system, to deactivate the appropriate active redeliver functions. =============================================================== 2007-09-03 DocDeliver 3.0.0 =============================================================== + IMPORTANT: DDS 3.0 by default uses an embedded XML database for storing job information. This means that the usual xml job files will no longer be present in the data/docdeliver/* directories and that manual manipulation of files in these directories is deprecated. Rather the new AXT JOBS GUI should be used for searching and resending files. A file based implementation is still available but deprecated since this implementation does not support more advanced searching of job files and information and thus usage will not support the AXT JOBS GUI. The used implementation is controlled by the docdeliver.persistance.implementation property in the docdeliver.properties file. The JOBS GUI is accessible from the top right menu bar in the AGETOR(r) Control Center. + DDS accepts checkpoint texts to be logged in case of delivery failure or success. The DDS parameters (passed to the outlet filter) controlling this are checkpointTextOnError and checkpointTextOnSuccess. + Delivery attempts do not produce an error but generates an info checkpoint. This functionality may be switched on/off by the docdeliver.checkpointsonretries property. The level of these checkpoints can also be set. + retryPattern attribute added to media configuration. The attribute may be used instead of the tentative and retryInterval attributes to specify a sequence of retry intervals. E.g. "30 sec;60 sec;5 min;1 hour" states that four retry attempts should be made with the given intervals. Default unit is seconds so "30;300;3600" would specify 30 seconds, 5 minutes and one hour between retries. + The default AXT URL in the AXT Media now refers to the AXT base address property (agetor.admin.axt.host) which hold the common configured base address of AXT on the machine. + Docdeliver now generates a checkpoint when doing a redelivery. + A new property enrichment system is present in DDS that allows for the enrichment of jobs with document and trace properties from the trace system such that these are copied to the jobs database and presented in the JOBS GUI. Enrichment may be controlled per job and is configured using the media configuration files and/or by DDS job properties. =============================================================== 2007-03-06 DocDeliver 2.2.1 =============================================================== + Added __ftpcmd.before* and __ftpcmd.after* after options to ftp media, to allow a list of ftp commands to be sent before or after an upload has taken place. + Improved support for received nested exceptions. =============================================================== 2007-01-15 DocDeliver 2.2.0 =============================================================== + FTP activemode property now available for FTP-media to control if active/passive communication is used + added Redeliver IDL interface methods. These methods allow remote componentes to request failed DDS and AXT job information. This facility is used from the central L&T service to display job information stored on other servers regardless of firewalls + Added serie.xml to package so initial error due to missing file no longer occur + Mal-formed job files that can not be read are moved to failed directory rather than retried over and over (causing loads of error logging) + Added the savefaileddata option to the DocDeliverOutlet Filter which tells the docdeliver not to save document and job data if the document fails. This also means the job will not be available for manual redelivery. + Added savejobdata option to the DocDeliverOutlet Filter which tells the docdeliver not to save job data, when the job succeeds. Ie. the docdeliver succeed catalogue will not contain a receipt for the succeeded job. + Added response support for the AXT media, using the output parameter. =============================================================== 2006-09-01 DocDeliver 2.1.1 =============================================================== + Logging subsystem refactored + added renameonupload option. Using this option resolves problems with ftpservers which doesn't allow rename, or which already provides a solution to file upload in progress issue. Default behaviour is set to true. =============================================================== 2006-08-08 DocDeliver 2.1.0 =============================================================== + IMPORTANT: Has been rebuild and packaged for use with AXT 2.1.0 and ADK 2.1.0. =============================================================== 2005-09-29 DocDeliver 2.0.11 --------------------------------------------------------------- + Bugfix: under heavy load from multiple queues, a file was sought delivered more than once causing error notification. + Bugfix: v. 2.0.10 media configuration files were not copied in properly during install =============================================================== 2005-08-25 DocDeliver 2.0.10 --------------------------------------------------------------- + Support for increased throughput and QOS by multiple simultaneous downloads and transformations. This multithreaded delivery adds robustness against non-responding media that no longer will lock up the outlet. + Multiple delivery queues allowing parallel delivery yet still serial delivery for jobs within a queue. By default jobs to same destination maps to same queue and are thus delivered sequentially. + Fine grained control over delivery properties per media type and destination. Particular servers (FTP, Mail etc.) may have special delivery properties (number of retries, time between retries, max simultaneous connections etc.) + Priority attribute for jobs allows for prioritization among jobs for different media/destinations. + Surveillance of processors to ensure their healthiness. + Configurable mail-notification in case of processor timeout (inactivity). + Surveillance of delivery directory. When number of files for delivery exceed a configured value, warning is sent. Should number of files drop below the threshold later, a cancellation is sent as well. + Improved status information in AGETOR® Control Center. + Specification of retry delay per destination. E.g. 5 retries with 10 minute interval for some specific FTP-server. + IMPORTANT: Added new media "axt" allowing delivery of data to AXT. Particular useful for redelivery of files from FTPInlet and MailInlet since these will now (in latest versions) store undeliverable files in DD-compatible format (.dat and .xml file). + IMPORTANT: Proprietary media plugins written for previous versions of the outlet must be rebuild to work with this new version. See the "what's new guide" accompanying this update for details. + IMPORTANT: The info file containing meta information on the delivery (Job_info_.xml) now has a slightly different format. This means that .xml files from previous vesions cannot be used with this version. You should make sure to clean up in the fail directory before upgrading. =============================================================== 2005-06-14 DocDeliver 2.0.9 --------------------------------------------------------------- + Bugfix: When multiple files to same server failed redelivery by placing .dat and .xml files in delivering directory did not work. + Better status information in AGETOR(r) Control Center including totals and list of resent events (errors, deliveries). + Number of attempts to delete a remote file was changed from 10 to 3. =============================================================== 2005-02-10 DocDeliver 2.0.8 --------------------------------------------------------------- + Bugfix: DocDeliver service now properly tries to connect to specified ftp server a configurable number of times before failing. + Added support for synchronized data delivery when invoking DocDeliver from AXT. I.e. the AXT DocDeliverOutlet filter will not return until the data handed to the DocDeliver outlet has completed delivery or failed. The option is enabled using the filter parameter: The default is not to synchronize delivery as hitherto. + errorMail and ftpadr attributes in the docdeliver configuration were renamed to smtpserver and ftpserver for uniformity (we still support the previous names though). =============================================================== 2004-10-25 DocDeliver 2.0.7 --------------------------------------------------------------- + Made the job data extension configurable (the default is .dat). This extension is used for naming files temporarily during upload and for storing the job data locally if this has been configured. =============================================================== 2004-10-13 DocDeliver 2.0.6 --------------------------------------------------------------- + IMPORTANT: This version requires AXT Basic 2.0.4. + IMPORTANT: DocDeliver has been updated, so it will try to automatically select the right fileparser when communicating with the FTP-server. That means that in most cases you no longer need to set the fileparser attribute on the ftp-folders in the configuration, just remove it. In some cases the fileparser needs to be set to the correct one. + Added a new fileparser, LinuxFTPFileListParser that suits most FTP servers on Linux and also some servers on AIX. Please refer to the user guide for more information. + The version number will now following the Agetor Install tool versioning. + It has been updated to use the updated fileparsers in AXT Basic. + Bugfix: When using the file system as media the attribute "filetype" was ignored. The attribute can be used to transfer the file as either "text" or "binary". + Jobs to the same server will now be sorted according to the time that they where sent to DocDeliver. That means that the jobs will be delivered in the same order that they was received. =============================================================== 2004-03-15 DocDeliver 2.0.5 --------------------------------------------------------------- + Bugfix: Removed a dependency to JDK 1.4 in the mail media + Bugfix: Fixed a bug where the email was not send if the email-address was not valid by the RFC822 standard. Email- address like "[fax:12121212]" should now be send to the mail- server. + In the date formats used in rich names you can now use all characters in the separation. This means that in the “folderdateformat” parameter in the file system and FTP media you can use “/” character to set more than one folder. + In the FTP media it is now possible to send FTP site commands to the FTP-server after the file has been handled by AXT or when an error occurs. =============================================================== 2004-01-19 DocDeliver 2.0.4 --------------------------------------------------------------- + IMPORTANT: The value of the series and the job id is now save in an XML-document so it will continue with the next number when DocDeliver is restarted. Before DocDeliver would start from the beginning every time it has started. + IMPORTANT: When a folder doesn't exist on either the file system or FTP media DocDeliver will try to create the folders. It is also possible to use rich names in the folder names as in the filenames. + IMPORTANT: The possibility to use the file system as a new media with the same functions as the FTP media has been added. + Fixed a bug where the timeout value of the FTP media was set to a very high value causing the timeout period to be much longer than set in the configuration. + Added the possibility to override a file where the filename on either the file system or FTP media has the same name as the one being delivered. This can be controlled with a new parameter, but the default action is to give the new file a different filename. + The resolving of rich names in file system, FTP and mail media is now the same. It is now possible to use milliseconds in the date format. + Fixed a bug where the FTP media failed to upload the files to the correct filename. To solve the problem there has now been added a new parameter to control the file parsing on the specific platform. + The email-address of the mail media can now be none correct email-address. Some mail-servers can handle delivering a mail as a fax if you send a email to something like "[fax:90909090]" which is not a correct email-address. DocDeliver will no longer check the syntax of the email-address. + Added the possibility to save the data document in the succeed folder together with the job description. This is control with the use of a parameter. The default behaviour is to delete the data document if the document was successfully delivered to the media, as in the previous versions of DocDeliver. As always, DocDeliver will save the data document in the failed folder if the document couldn't be delivered to the specified media. + Some old classes have been removed and the log has been improved. =============================================================== 2003-10-31 DocDeliver 2.0.3 --------------------------------------------------------------- + It is now possible to set the maximum number of characters that a serie will be in the filename used for FTP =============================================================== 2003-09-16 DocDeliver 2.0.2 --------------------------------------------------------------- + The package was missing a jar-file for the FTP media =============================================================== 2003-08-28 DocDeliver 2.0.1 --------------------------------------------------------------- + Minor bug fix in error mail dispath when file delivery fails. It is possible to specify name for mailserver username and password to be used when sending an error mail. =============================================================== 2003-05-30 DocDeliver 2.0.0 --------------------------------------------------------------- + IMPORTANT: The value of the properties brokerTimeout and timeout has changed to be in seconds instead of milliseconds. Please consult the AXT DocDeliver user guide for explanation of the two different properties and how to use them. + IMPORTANT: Has been rebuild and packaged for use with AXT 2.0.0 and ADK 2.0.0. =============================================================== 2002-05-02 DocDeliver 1.0.1 --------------------------------------------------------------- + IMPORTANT: This version of AXT DocDeliver requires ADK 1.3.7 or newer. + IMPORTANT: All classes recompiled according to ADK 1.3.7 + Choose between ASCII and Binary File mode in FTP transfers, ensuring a correct download of the files to the ftp-server. + Is now cleaning up temporary files that didn't finished being uploaded to AXT DocDeliver before the service was stopped the last time. + Automatically starting to deliver any waiting jobs when AXT DocDeliver is started from the servicerunner. + Use the new servicerunner to see status information about AXT DocDeliver, the file queues and the current processing. + Stopping the service with the new servicerunner ensures the file processing to be finished correctly and nicely before stopping AXT DocDeliver.