AW Questions and AW Audit Resolution

[Alison Alter]

Dear Colleagues,

I look forward to our continued discussion of the recent boil water situation and the vote on my Austin Water Audit Resolution at tomorrow’s meeting. The resolution is posted as item 60 and can be found here: https://www.austintexas.gov/department/ ... eg.htm#060. I appreciate the co-sponsorship of Council Members Tovo, Fuentes, Pool, Ellis, and Kelly.

As requested, I am posting additional questions for Austin Water in advance of the meeting.

Broadly speaking, I believe we need to better understand the operational system errors of the organization so we can fix them and figure out what resources we need to invest to prevent future water quality and supply failures. I want to understand if we are following best practices for our manual operations and automation levels, whether we need to update SCADA, whether we need SOP revisions, whether we need to change treatment approaches, whether we need to recommend updates to TCEQ regulations.

Here are some specific questions that I would like answered as soon as possible:

1. Why did staff keep adding the processed solids / lime longer than they should have?

2. How did the increased settled water turbidity go unnoticed and how did the filtered water turbidity go unnoticed for so long?

3. What kind of sensors do we have in place to monitor the turbidity in the open channel settled water trough - which connects the basin to the water filtration area? Can the processed solid additions be automatically turned off if the turbidity is high or trending high in the settled water troughs?

4. Explain what kind of automation we have in place for bringing a basin online. If that process is not automated, explain why we don’t automate it. If automated, what kind of standard operating procedures (SOPs) do we have for manually checking for settled water and filtered water turbidity? Are SCADA controls involved in basin seeding and filling? Do the alarms go off to multiple people from the SCADA system? If the alarms are not answered, do supervisors get notified of the situation? Were any automated processes bypassed or turned off? Was this a failure in automation (SCADA)? If so, how do we fix it?

5. How many operators were working when the failure occurred and what is their level of licensing? Was the number of engineers working the recommended number?

6. Were SOP's followed? If not, why not?

7. Please provide a list of all internal plant alarms for the week prior to the failure. Were there process alarms that notified someone of the issue, and if not, why not?

8. Chain of command notifications: How quickly was the shift supervisor notified? How quickly was maintenance notified? How quickly was the head of water notified?

9. Equipment Failures: Were there equipment failures that led to process failure? Was that equipment part of a checklist that operators verify/validate? Please provide the daily, weekly, and monthly operator checklists. When was the last time dosing pumps, sensors, etc. associated with the failure were calibrated?

10. What steps if any were taken to isolate the contaminated water? Was there an attempt to divert water back to the head of the plant? Is it possible to divert contaminated flows to storage basins and or back to the head of the plant?

11. Are there filter beds on the back side of the process? Why did these fail? Were some of them offline? Why does the city not have sufficient filtration redundancies in place to manage process upsets?

12. What are you doing this week to make sure this can’t happen again? What has already changed? What are the business systems that need to be put in place to make sure this never happens again? - manpower, automated systems, treatment changes etc.

13. The past 3 boil water notices and the taste and odor issue from the zebra mussels were due to operational and asset management issues at Ullrich. What has been done to improve the operations and communication at Ullrich? What type of asset management plan is in place at Ullrich to ensure it does not happen again?

14. Do we have SCADA or other automation failures happening within our treatment plants?

15. When was the last time that we hosted a peer review by high performing US water utilities or related associations to identify areas of needed improvement to our practices and facilities?

I am looking forward to our discussion tomorrow and in the coming months and weeks.

Regards,
Alison Alter

[Sabino Pio Renteria]

Thank you for raising these important questions, Council Member Alter.

As I mentioned during yesterday’s meeting, I would also like to get information about staff attrition at Austin Water and if that is contributing to the operational problems we have been seeing.

More specifically:

1) What is the current rate of attrition (by role/position) at AW? How does that compare to other similar departments?

2) Are attrition/retention issues contributing to the problems that have been occurring? If so, how?

3) If we are seeing retention issues, especially in critical roles, why?

4) Is recruitment of qualified staff an issue?

5) Any potential measures that may be needed to address attrition/retention/recruitment issues, and the fiscal impact of those measures.

-Pio

[Steve Adler]

Thank you colleagues for posting your questions here. I look forward to our conversation tomorrow.

Again, this is a tremendous start to the work ahead. One question of mine that remains unanswered from Tuesday is what risk would there have been to the community HAD there been bacteria released in the plants prior to the issuance of the boil water notice? What was the potential risk in our timeline?

-S