Multi-modal Large Model Driven Whole House Decoration KnowledgeQuestion and Answer and Scheme Automatic Generation

Abstract

Focusing on the pain points existing in traditional decoration services, such as asymmetric information, vague demand
expression and high iterative cost of schemes, this paper puts forward a whole house decoration knowledge quiz and scheme automatic generation system based on multi-modal large model. The system adopts a hierarchical micro-service architecture,covering four core modules: multi modal input understanding, knowledge and intention center, scheme generation and optimization, and presentation and interaction. In the process of multi modal input understanding, by integrating heterogeneous inputs such as text, sketch, photo and voice, with the help of advanced technologies such as large language model, visual Transformer and graph neural network, the user's spatial constraints, style preferences and functional requirements are accurately analyzed, and a unified semantic representation is generated. The knowledge and intention center combines the domain knowledge base to deeply analyze and refine the user's intention, which provides strong support for the scheme generation. The scheme generation and optimization layer automatically generates a complete decoration scheme including 3D layout, bill of materials and budget estimation based on the parsed user intention and knowledge base information, and supports dynamic adjustment and optimization. The experimental results show that the system is superior to the traditional manual design and single-mode and two-mode systems in terms of intention accuracy, constraint satisfaction and material recommendation performance. The generation time of the first draft of the scheme is greatly shortened, the cost of a single iteration is reduced to zero, and the user satisfaction is improved.