Platform and campaigns issue
Incident Report for ACTITO
Postmortem

Cher Utilisateur Actito

‌Nous avons rencontré ce vendredi un incident ayant affecté l’utilisation d’ACTITO pour certains utilisateurs.

Nos équipes ont immédiatement pris l’incident en charge.

La panne a rapidement pu être identifiée comme étant un problème de réseau nécessitant une intervention physique dans un de nos datacenter.

Malheureusement, en raison des règles de confinement, l’intervention n’a pu se dérouler aussi rapidement que normal dès lors que les accès aux datacenters étaient limités.

Lorsque le datacenter a été rendu accessible, nos équipes ont pu réaliser leur intervention. Ils ont mené des tests extensifs, effectué le remplacements hardware nécessaires et réinitialisé les différents équipements touchés. La panne a ains pu être solutionnée courant de l’après-midi.

Un monitoring approfondi et des tests complémentaires ont été effectués en cours de week end pour s’assurer à ce que l’incident ne se reproduise plus.

Nous vous prions d’excuser les inconvénients que cette situation a pu vous causer.

Bien cordialement

Posted Apr 06, 2020 - 18:09 CEST

Resolved
This incident has been resolved.
Posted Apr 06, 2020 - 18:01 CEST
Monitoring
The access to the platform has been restored and all applications are now operational again.
Campaigns are correctly generated and sent. All email links are working.

Minor delays can be still experienced with reporting. No interactions are lost.

Additional maintenance operations will be carried out at night during the weekend (between 11pm and 4am)
Posted Apr 03, 2020 - 16:41 CEST
Identified
Access to the platform is now very unstable again.

Our IT team is working actively on restoring access.
Posted Apr 03, 2020 - 15:38 CEST
Update
The delay on e-mail campaigns has been caught up: all campaigns have been sent correctly.
Some issues are still possible when clicking on email links, these problems are diminishing quickly : less than 5% clicks now encounter errors.

Major delays are however encountered in reporting : the number displayed in the interface are not up to date. They are gradually updated. No interactions are lost.

Our IT team is still working on monitoring and stabilizing the platform
Posted Apr 03, 2020 - 15:07 CEST
Update
We are continuing to monitor for any further issues.
Posted Apr 03, 2020 - 13:41 CEST
Monitoring
A number of application have been successfully restarted :
Targeting are now calculated and e-mail campaigns are being recovered and sent.
More than 90% of the links in e-mail campaigns are working correctly.

The access to the platform remain slow and you may still experience issues with pages loading.
Our IT team is now working on restoring platform access.
Posted Apr 03, 2020 - 13:39 CEST
Identified
The origin of the issue has been identified and our IT team is working on fixing it.
We are keeping you updated as soon as possible.
Posted Apr 03, 2020 - 11:05 CEST
Investigating
We are currently experiencing slowdown issues.
Access to the platform is slowed and pages may take longer to load.
Campaigns are slowed and the sending speed is decreased. Links of sent campaigns may experience issues.

Our IT team is working on improving performances. We are monitoring every campaign in order to guarantee its sending as quickly as possible.
The slowdown also caused some synchronization to fail. We are working to relaunch them.


We are keeping you up to date on the situation evolution as soon as possible.
Posted Apr 03, 2020 - 09:10 CEST
This incident affected: Portal, Email campaign, and Reporting.