Set the Dutch oven over medium-high heat and add the olive oil. When it shimmers, add the sliced andouille. Cook for 4 minutes, stirring occasionally, until the edges brown and the pot has some fond (those browned bits) on the bottom.
Add the onion, celery, and bell pepper. Cook for 5 minutes, stirring and scraping as the vegetables soften and pick up all that smoky sausage flavor. Add the garlic and cook 30 seconds, just until fragrant.
Stir in the tomato paste, Cajun seasoning, smoked paprika, thyme, cayenne, salt, and black pepper. Cook 1 minute to wake up the spices and lightly caramelize the paste (it should look brick-red and smell toasty).
Sprinkle the 3 tablespoons flour over everything and stir well for 1 minute. You're not making a dark roux here, just cooking out the raw flour taste so the broth thickens smoothly.
Slowly pour in the chicken broth while stirring to prevent lumps, then stir in the milk. Add the chicken thigh pieces and the bay leaf (if using). Bring to a gentle simmer over medium heat.
Reduce heat to medium-low and simmer, uncovered, for 12 minutes, stirring every couple minutes so the milk doesn't scorch. The broth should look slightly thickened, like a light gravy. If it's boiling hard, turn it down; aggressive boiling can toughen the chicken and break the dairy.