Put the potatoes in a pot and cover with water. Cover and bring to a boil. Reduce the heat to a simmer and cook approximately 10 minutes until the potatoes are tender to a knifepoint, but not overly soft. Remove from the heat, drain and cool enough to handle. Peel and cut them into thick slices.
While the potatoes are cooking, make the sauce by putting all the ingredients (evaporated milk, queso fresco, crackers, oil and aji amarillo) in a blender or food processor and blend until smooth.
Put some lettuce on serving plates, put potato slices on top and pour over some of the sauce (you can use more/less to taste). Top with slices of olive and some slices of hard boiled egg on the side.
Notes
You can cook the potatoes ahead and also make the sauce ahead if needed. Note potatoes can become a little watery if stored too long in the fridge, so better, if possible, is to cook and store at room temperature for an hour or so. The sauce freezes well and you can make extra/store leftover to use as a dip or sauce for other things (see above).This recipe makes approx 1 cup/240ml of sauce.