In a large skillet, heat the olive oil over medium-high heat. Add the remaining 2 slices ham and cook until its fat begins to render and its edges begin to approach crispiness. Remove and set aside. Add one or two trout to the pan and cook for a few minutes on each side, until the fish gains a bit of color. Transfer to a baking sheet and repeat to cook the remaining fish.