151 lines
5.4 KiB
Plaintext
151 lines
5.4 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>tag</th>\n",
|
|
" <th>title</th>\n",
|
|
" <th>body</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>功能文档</td>\n",
|
|
" <td>数据表格</td>\n",
|
|
" <td>数据表格 数据表格是简化版的excel，可以使用【数据表格】相关的指令对其进行操作，支持导...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>问答</td>\n",
|
|
" <td>数据表格数据处理</td>\n",
|
|
" <td>数据表格批量获取数据后，然后循环处理数据表格，再循环里面过滤不要的数据，用删除数据表格行删除...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>2</th>\n",
|
|
" <td>指令文档</td>\n",
|
|
" <td>清空数据表格</td>\n",
|
|
" <td>清空数据表格 清空数据表格 注：【 清空数据表格 】指令一般主要是用于在流程开始前初始化数...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>3</th>\n",
|
|
" <td>指令文档</td>\n",
|
|
" <td>数据表格导入</td>\n",
|
|
" <td>数据表格导入 导入数据到数据表格 文件路径 目标文件的绝对路径,支持xls、xlsx、cs...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>4</th>\n",
|
|
" <td>指令文档</td>\n",
|
|
" <td>数据表格导出</td>\n",
|
|
" <td>数据表格导出 将数据表格导出为文件 文件夹 桌面/下载目录/临时目录/自定义目录 文件名称...</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>"
|
|
],
|
|
"text/plain": [
|
|
" tag title body\n",
|
|
"0 功能文档 数据表格 数据表格 数据表格是简化版的excel，可以使用【数据表格】相关的指令对其进行操作，支持导...\n",
|
|
"1 问答 数据表格数据处理 数据表格批量获取数据后，然后循环处理数据表格，再循环里面过滤不要的数据，用删除数据表格行删除...\n",
|
|
"2 指令文档 清空数据表格 清空数据表格 清空数据表格 注：【 清空数据表格 】指令一般主要是用于在流程开始前初始化数...\n",
|
|
"3 指令文档 数据表格导入 数据表格导入 导入数据到数据表格 文件路径 目标文件的绝对路径,支持xls、xlsx、cs...\n",
|
|
"4 指令文档 数据表格导出 数据表格导出 将数据表格导出为文件 文件夹 桌面/下载目录/临时目录/自定义目录 文件名称..."
|
|
]
|
|
},
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"import requests\n",
|
|
"url = 'https://api.yingdao.com/api/noauth/v1/sns/fullIndex/search'\n",
|
|
"headers = {\n",
|
|
" 'accept': 'application/json',\n",
|
|
" 'authorization': 'Bearer 4e49960a-4d22-4fed-acea-d6faee687c4c',\n",
|
|
"}\n",
|
|
"\n",
|
|
"all_data = []\n",
|
|
"page = 1\n",
|
|
"while True:\n",
|
|
" params = {\n",
|
|
" 'page': page,\n",
|
|
" 'size': 20,\n",
|
|
" 'key': '数据表格',\n",
|
|
" 'source': ''\n",
|
|
" }\n",
|
|
" response = requests.get(url, headers=headers, params=params)\n",
|
|
" data = response.json()\n",
|
|
" if not data['data']:\n",
|
|
" break\n",
|
|
" all_data.extend(data['data'])\n",
|
|
" page += 1\n",
|
|
"\n",
|
|
"all_data\n",
|
|
"import re\n",
|
|
"\n",
|
|
"def remove_em_tags(text):\n",
|
|
" return re.sub(r'<em>(.*?)</em>', r'\\1', text)\n",
|
|
"\n",
|
|
"import pandas as pd\n",
|
|
"# Extract tag, title, and body from the results\n",
|
|
"extracted_data = []\n",
|
|
"for item in all_data:\n",
|
|
" extracted_data.append({\n",
|
|
" 'tag': item.get('tag'),\n",
|
|
" 'title': remove_em_tags(item.get('title', '')),\n",
|
|
" 'body': remove_em_tags(item.get('body', ''))\n",
|
|
" })\n",
|
|
"# Create a DataFrame and save to an xlsx file\n",
|
|
"df = pd.DataFrame(extracted_data)\n",
|
|
"df.to_excel('extracted_data.xlsx', index=False)\n",
|
|
"df.head() # Display the first few rows of the DataFrame to verify\n"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.12"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|