# 1. 多模态大模型简介 # 1.1 Qwen2.5-Omni Qwen 2.5-Omni 系列由阿里通义千问团队开源，包含 Qwen 2.5-Omni-3B 与 Qwen 2.5-Omni-7B 两个参数规模。该系列模型统一支持文本、音频、图像、视频四大模态输入，并可实时输出文本与自然语音。
在多模态基准 OmniBench 上，Qwen 2.5-Omni 系列取得新 SOTA，整体表现超越 Gemini-1.5-Pro 等同类模型。在单模态任务中，包括语音识别、翻译、图像推理、视频理解与语音生成，Qwen 2.5-Omni 也全面领先同等规模的单模态和闭源模型。其中 7B 版本在 seed-tts-eval 语音生成基准中展现出接近人类的语音合成能力。
项目地址：https://github.com/QwenLM/Qwen2.5-Omni (opens new window) 模型地址：https://huggingface.co/collections/Qwen/qwen25-omni-67de1e5f0f9464dc6314b36e (opens new window)
# 1.2 Qwen2.5-VL Qwen2.5-VL 系列由阿里通义千问团队推出，涵盖 3B、7B、32B、72B 四个规模。模型在 图像解析、内容识别、视觉推理 等任务上表现突出。其中 Qwen2.5-VL-32B-Instruct 经过强化学习优化，在人类偏好对齐和数学推理上显著提升，性能甚至超越上代的更大规模模型，如 Qwen2-VL-72B-Instruct。
项目地址：https://github.com/QwenLM/Qwen2.5-VL (opens new window) 模型地址：https://huggingface.co/collections/Qwen/qwen25-vl-6795ffac22b334a837c0f9a5 (opens new window)
# 2. 多模态大模型请求方式 这是以阿里百炼平台的Qwen-Omni多模态模型为例，详见：https://help.aliyun.com/zh/model-studio/qwen-omni#b4d9c9c405qwg (opens new window)
# 2.1 请求图片模态 $ curl -N https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer sk-xxx" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-omni-turbo",
"messages": [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"
}
},
{
"type": "text",
"text": "图中描绘的是什么景象？"
}
]
}
],
"modalities": ["text"],
"stream": true,
"stream_options": {
"include_usage": true
}
}'
12345678910111213141516171819202122232425262728如果要以base64的方式传参，详见：https://help.aliyun.com/zh/model-studio/qwen-omni#c516d1e824x03 (opens new window)
# 2.2 请求音频模态 $ curl -N https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer sk-xxx" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-omni-turbo-0119",
"messages": [
{
"role": "user",
"content": [
{
"type": "input_audio",
"input_audio": {
"data": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250211/tixcef/cherry.wav",
"format": "wav"
}
},
{
"type": "text",
"text": "这段音频在说什么"
}
]
}
],
"modalities": ["text"],
"stream": true,
"stream_options": {
"include_usage": true
}
}'
1234567891011121314151617181920212223242526272829# 2.3 请求视频模态 $ curl -N https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer sk-xxx" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-omni-turbo",
"messages": [
{
"role": "user",
"content": [
{
"type": "video_url",
"video_url": {
"url": "https://samplelib.com/lib/preview/mp4/sample-5s.mp4"
}
},
{
"type": "text",
"text": "请详细描述这个视频的内容"
}
]
}
],
"modalities": ["text"],
"stream": true,
"stream_options": { "include_usage": true }
}'
1234567891011121314151617181920212223242526# 3. 多模态大模型实测对比 对比型号：Qwen2.5-Omni-7B vs Qwen2.5-VL-32B-Instruct
# 3.1 手写体识别任务 # 3.1.1 测试结果 Prompt: 请识别出这幅图片里的文字。
# 3.1.2 测试评价 根据手写体识别的样例来看，Qwen2.5-Omni-7B的手写体识别准确率很高，即使对于潦草的字迹也能准确识别。
# 3.2 雅思作文打分任务 # 3.2.1 测试题目 作文题目：
You should spend about 40 minutes on this task. Write about the following topic:
Shopping is becoming more and more popular as a leisure activity.However, some people feel that this has both positive and negative effects.
Why is shopping so popular? What effects does its increase in popularity have on individuals and on society? Give reasons for your answer and include any relevant examples from your own knowledge or experience. Write at least 250 words.
12345作文示例：
打分参考：
参考评分：8分
参考评价：All parts of the task are sufficiently and appropriately addressed. A clear position is presented and supported and ideas are relevant and well-developed. Some further extension in supporting points, such as some mention of negative effects, however brief, could be added to raise this to the highest band for Task Response. There is a clear progression throughout the response and a wide range of cohesive devices is used with flexibility, although there is evidence of rare errors too. However, the personal comment in the final paragraph is a sudden topic shift after the general exploration of the issues. This is a slight lapse in the development and clear progression of the response, and prevents it from reaching the highest band for Coherence and Cohesion. A wide range of vocabulary is used with flexibility and precision. The occasional minor error, such as the omission of dependent prepositions, is evident but not systematic. Similarly, minor errors or omissions occur in structures, but this does not detract from the wide range flexibly and accurately used in general; the majority of structures are error-free.
12# 3.2.2 测试提示词 雅思写作评分要求主要涉及四个方面：任务回应（task response）、连贯与衔接（coherence and cohesion）、词汇资源（lexical resource）以及语法范围和准确性（grammatical range and accuracy）。对于任务回应，考官会看考生是否完整地回答了题目要求的内容，是否涵盖了所有要点，是否对题目进行了充分的展开和讨论。在连贯与衔接方面，主要考察文章的结构是否清晰合理，段落之间是否有逻辑关系，句子之间是否使用了合适的连接词和过渡句来使文章流畅自然。词汇资源的评分则关注考生是否使用了丰富多样的词汇来表达观点，是否能够准确地运用词汇，避免重复使用同一词语。语法范围和准确性方面，考官会检查考生是否使用了多种语法结构，包括简单句、复合句等，以及这些语法结构是否正确无误，是否有语法错误影响了文章的理解。
现有雅思作文题目要求为：You should spend about 40 minutes on this task. Write about the following topic: Shopping is becoming more and more popular as a leisure activity.However, some people feel that this has both positive and negative effects. Why is shopping so popular? What effects does its increase in popularity have on individuals and on society? Give reasons for your answer and include any relevant examples from your own knowledge or experience. Write at least 250 words.
给定一篇关于该题目的作文图片，请完成任务：
1. 识别出图片中作文的文本内容。
2. 给出评分意见并打分，满分为10分。
1234567# 3.2.3 测试结果 Qwen2.5-Omni-7B：
嗯…首先呢，这篇作文的任务回应还算不错，基本覆盖了题目要求的内容，提到了购物受欢迎的原因和对个人和社会的影响。不过呢，在连贯与衔接上有点问题，有些地方句子之间的连接不够紧密，比如“shopping is very popular among the majority of the population”后面直接就转到原因，中间缺少过渡。词汇资源方面，虽然有一些简单的词汇，但整体感觉比较单一，没有太多高级词汇或者多样化的表达。语法范围和准确性也不错，大部分语法结构都用得挺好的，只是个别小错误。综合来看，我觉得能有6分吧。你要是还有类似的作文想聊聊，随时跟我说哈。
1Qwen2.5-VL-32B-Instruct：
为了完成任务，我们需要首先识别出作文的文本，并对其进行分析，然后根据雅思写作的评分标准（任务回应、连贯与衔接、词汇资源、语法范围与准确性）给出评分意见及打分数。
---
#### **第一步：识别作文文本**
以下是作文的文本（已根据图像整理）：
**正文：**
Shopping is very popular among the majority of the population, despite their age or social standards. This is due to the fact that most people find shopping as an experience which makes a difference in their day to day life. Some people find themselves in a different world, surround by various material things which they may or may not be able to purchase. Even if they are not able to afford these, they find much more pleasurable in looking and feeling these things and imagining owning them some day. Some people go shopping when they really want to purchase things and they prefer to look around and compare the prices before buying them. I think most people love to go shopping because, then they could meet friends, make contacts with people, get to know what is in the latest trend. It provides them a pleasurable distraction from their daily life and sometimes dulle life.
The most important effect shopping has on an individual is that it nurtures the self-esteem. An individual may find the social contract most gratifying and this makes people come out of their shell and be more extrovert. Purchasing something, as well as, it is an interesting and pleasurable experience, as each year passed by, simply because it enriched my heart and mind.
The increase in the population to go shopping has many effects on the society. People may be encouraged to go into new businesses and it will improve the economy. More job opportunities will be created for people in these businesses. Encouraged by the increase of population, the increase of people will try to improve their products, so that the consumers can have a wide range of products at a better price. The overall quality of the product as well as the services provided by the sales staff will be improved.
More job opportunities will be created in advertising as the manufactures try to sell their products. Consumers will have a better knowledge of the product so that they will not be misled by inferior quality products.
I find shopping an interesting and pleasurable experience, as each year passed by, simply because it enriched my heart and mind.
---
#### **第二步：评分分析**
根据雅思写作评分标准，我们将逐条分析这篇作文的表现。
##### **1. 任务回应 (Task Response)**
- **优点：**
- 作文总体回答题目要求，讨论购物为何受欢迎，并分析购物的个人和社会影响。
- 作者提到购物是一种体验，可以带来乐趣，帮助人们社交，并且分析购物如何影响个人自信和社会经济。
- **缺点：**
- 对题目中的“负面效应”讨论不足。题目要求讨论购物的“正面”与“负面”影响，但文中几乎没有提到购物的负面效应（例如过度消费、环境问题等）。
- 部分类似句子重复，例如“pleasurable”多次出现，缺乏深度展开。
- **评分建议：**
作者基本回答题目，但缺乏全面性和深度，特别是没有讨论负面效应。因此，任务回应部分表现一般。
**评分：6/9**
##### **2. 连贯与衔接 (Coherence and Cohesion)**
- **优点：**
- 文本有一定的结构，分为三个部分：购物受欢迎的原因、个人影响、社会影响。
- 使用了一些连接词，例如“this is due to the fact that”、“as well as”、“so that”等，有助于段落之间的衔接。
- **缺点：**
- 段落之间逻辑关系不够清晰，特别是第二段提到“pleasurable distraction”时，逻辑略显跳跃。
- 一些句子结构松散，例如“as well as”使用不当，导致句子不够流畅。
- **评分建议：**
文本有一定的连贯性和衔接，但段落间的逻辑关系不够紧密，部分句子结构松散。
**评分：6/9**
##### **3. 词汇资源 (lexical resource)**
- **优点：**
- 使用了一些较高级的词汇，例如“self-esteem”、“gratifying”、“extrovert”等，显示出一定的词汇多样性。
- 作者尝试使用了一些较复杂的表达，例如“pleasurable distraction”、“encouraged by the increase of population”。
- **缺点：**
- 词汇重复现象明显，例如“pleasurable”多次出现。
- 一些词汇使用不够准确，例如“social contract”可能用词不当，此处应使用“social contact”。
- **评分建议：**
词汇有一定的多样性，但重复现象明显，且部分用词不够准确。
**评分：6/9**
##### **4. 语法范围与准确性 (Grammatical range and accuracy)**
- **优点：**
- 文本包含多种句式的使用，例如复合句、简单句等。
- 基本语法结构正确，没有严重的语法问题。
- **缺点：**
- 存在一些语法问题，例如“as each year passed by”时態使用不当，应改为“as each year goes by”。
- 一些句子结构不够严谨，例如“encouraged by the increase of population”结构松散，导致语义模糊。
- 个别拼写问题，例如“social contract”应改为“social contact”。
- **评分建议：**
语法范围较广，但存在一定语法问题，影响表达的准确性。
**评分：6/9**
#### **第三步：综合评分**
综合以上分析，这篇作文在任务回应、连贯与衔接、词汇资源、语法范围与准确性四个方面表现较为平均，但缺乏深度讨论，且在语法、词汇使用方面存在一定问题。整体表现尚好，但仍有提升空间。
**最终评分：**
- **任务回应 (Task Response): 6/9**
- **连贯与衔接 (Coherence and Cohesion): 6/9**
- **词汇资源 (lexical resource): 6/9**
- **语法范围与准确性 (Grammatical range and accuracy): 6/9**
**总评：**
综合评分：**6.0/9**
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182# 3.2.4 测试评价 Qwen2.5-VL-32B-Instruct有较好的指令遵循能力，能够同时完成作文识别、打分、给出意见的任务，并且生成内容格式较好能够分点罗列。Qwen2.5-Omni-7B的指令遵循能力较弱，无法完全根据指令生成回复，同时生成内容口吻较为口语化，并且没有分点罗列。
# 3.3 雅思口语识别任务 # 3.3.1 音频识别 Prompt：请识别出上述对话中的文本内容，并标记不同对话人物。不要做总结，你只需要展示对话内容。
Qwen2.5-Omni-7B返回结果：
Man: Alright let's begin. So could you tell me your full name please? Woman: Hi my name is Yoon Deli and I go by the name Yoon J. Man: Okay. And Yoon J. Are you working or are you a student? Woman: I'm actually both. I'm currently attending Gonggu University and I'm majoring in stem cells and I'm also doing a part - time job at various places. I'm doing interviews for new members at an English Academy and I'm also teaching middle school students English. Man: And could you tell me about your hometown? Woman: I lived in Mapu when I was young near home university and it was a loud very lively place but since I was a young girl myself I really liked that town so that's one of the reason I'm visiting there very often. So like I said I work at the English Academy and actually that English Academy is near home there and that's one of the reason I chose to work there. The hometown I'm living right now at the moment is in Council near Parson Station. It's not their lively or loud as the place I used to live but it has lots of great food and stores so yeah I really enjoy living there. Man: Okay. Now we'll talk about collecting things. So do you collect anything? Woman: I actually collect photo cards. You know like when you buy an album there are either one or two photo cards and I really like those exclusive photo cards so that's actually one of my hobbies.如果还有关于这个对话的其他问题或者想继续聊聊，随时告诉我呀。
1评价：识别不是很精准，识别出来的文本不适合阅读。
# 3.3.2 发音纠正 Prompt：请你纠正上述女生的发音问题，生成一个发音标准的美式英语。
Qwen2.5-Omni-7B返回结果：
嗯…我觉得她发音挺准的呢。不过要是非说有啥问题的话，“gonggu”这个词听起来有点怪，应该是“Gonggu”。还有“Council”，这个音也得注意下。其他的像“Mapu”“home university”这些词发音都挺标准的。你是不是觉得我纠正得不对啊？可以和我说说你的想法哦。
1# 3.3.3 英音/美音生成 Prompt：请你纠正上述女生的发音问题，生成一个发音标准的英式英语音频。不要生成其它内容，直接生成英式英语的音频。
Qwen2.5-Omni-7B返回结果：
I'm sorry, I can't directly generate an audio file here. But if you want to correct your pronunciation in English, you could try listening to native speakers on platforms like YouTube or BBC Learning English. They have great resources for improving your accent and pronunciation.If you have any other questions about English learning or anything else, feel free to let me know.
1评价：模型似乎无法准确生成英音或者美音。模型生成的文本与音频似乎是一致的，就是把生成的文本转成音频再输出一次。
# 4. 多模态大模型选型建议 Qwen2.5-Omni-7B 与 Qwen2.5-VL-32B-Instruct 模型在不同人物上各有千秋，但整体而言，我更倾向于选择 Qwen2.5-Omni-7B。该模型参数量较小，计算开销低，推理速度更快，同时支持全模态输入输出。在实际使用中，Qwen2.5-VL-32B-Instruct 曾出现过“复读机”问题，而 Qwen2.5-Omni-7B 我暂时未遇到类似问题，整体表现更为稳定。
对比项 Qwen2.5-Omni-7B Qwen2.5-VL-32B-Instruct 参数量 7B 32B 计算开销 较低 较高 推理速度 较快 较慢 模态支持 支持全模态输入输出（文本、图片、音频、视频） 只支持文本与图片模态 指令遵循 一般 较好 稳定性 实测稳定，未发现“复读机”现象 实测曾出现“复读机”问题 适用场景 全模态需求、轻量化部署、低延迟场景 对视觉理解要求高、可容忍更大算力和延迟的场景 # 5. 参考资料 [1] Qwen2.5-VL-32B: 更聪明、更轻量! from 官方文档 (opens new window)
[2] 7B模型搞定AI视频通话，看听说写全模态打通 from 微信公众号 (opens new window)
[3] Qwen2.5-Omni 是阿里云 Qwen 团队开发的端到端多模态模型 from Github (opens new window)
[4] Qwen2.5-VL 是 Qwen 系列中最强大的视觉语言模型 from Github (opens new window)