In this paper, we focus on the structured multi-frame vectors in Hilbert $C^*$-modules. More precisely, it will be shown that the set of all complete multi-frame vectors for a unitary system can be parameterized by the set of all surjective operators, in the local commutant. Similar results hold for the set of all complete wandering vectors and complete multi-Riesz vectors, when the surjective operator is replaced by unitary and invertible operators, respectively. Moreover, we show that new multi-frames (resp. multi-Riesz bases) can be obtained as linear combinations of known ones using coefficients which are operators in a certain class.