feat: 初始化营销系统数据工具项目
- 添加基础项目结构，包括后端Go代码和前端静态文件 - 实现核心功能模块：数据导出、模板管理、元数据查询 - 添加多数据源支持（营销系统、易码通、元数据库） - 实现CSV和Excel导出功能 - 添加配置管理系统，支持YAML和环境变量 - 实现日志记录和请求追踪 - 添加Docker部署支持 - 编写README文档说明项目结构和启动方式
This commit is contained in:
commit
113a8ffa0a
|
|
@ -0,0 +1,246 @@
|
|||
# MarketingSystemDataTool 项目规则
|
||||
|
||||
## 项目概述
|
||||
|
||||
这是一个营销系统和易码通数据工具，提供数据导出、模板管理、元数据查询等功能。
|
||||
|
||||
### 技术栈
|
||||
- **后端**: Go 1.21，使用标准库 `net/http`，不使用第三方 Web 框架
|
||||
- **前端**: Vue 3 (通过 CDN 引入) + Element Plus
|
||||
- **数据库**: MySQL，支持多数据源（Marketing、YMT、Meta）
|
||||
- **导出格式**: CSV、Excel (使用 excelize/v2)
|
||||
- **配置**: YAML 配置文件 + 环境变量覆盖
|
||||
|
||||
## 项目结构
|
||||
|
||||
```
|
||||
server/ # Go 后端代码
|
||||
├── cmd/server/ # 主程序入口
|
||||
├── internal/ # 内部包
|
||||
│ ├── api/ # HTTP 路由和处理器
|
||||
│ ├── config/ # 配置管理
|
||||
│ ├── db/ # 数据库连接和池管理
|
||||
│ ├── exporter/ # 数据导出逻辑
|
||||
│ ├── logging/ # 日志工具
|
||||
│ ├── models/ # 数据模型
|
||||
│ ├── repo/ # 数据仓库层
|
||||
│ ├── schema/ # 数据库模式定义
|
||||
│ └── ymtcrypto/ # 加密工具
|
||||
web/ # 前端静态文件
|
||||
config/ # 非敏感配置
|
||||
scripts/ # 开发和运维脚本
|
||||
```
|
||||
|
||||
## Go 代码规范
|
||||
|
||||
### 架构模式
|
||||
|
||||
1. **Handler 函数模式**: 每个 API 端点使用 `Handler` 函数返回 `http.Handler`
|
||||
```go
|
||||
func ExportsHandler(meta, marketing, ymt *sql.DB) http.Handler {
|
||||
api := &ExportsAPI{meta: meta, marketing: marketing, ymt: ymt}
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
// 路由逻辑
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
2. **中间件链**: 使用函数式中间件，顺序为 `withAccess` -> `withTrace` -> Handler
|
||||
```go
|
||||
mux.Handle("/api/exports", withAccess(withTrace(ExportsHandler(...))))
|
||||
```
|
||||
|
||||
3. **Context 传递**: 使用 `context.Context` 传递 trace_id、SQL、请求元数据等
|
||||
- `TraceIDFrom(r)` - 获取 trace_id
|
||||
- `WithSQL(r, sql)` - 设置 SQL 到 context
|
||||
- `SQLFrom(r)` - 从 context 获取 SQL
|
||||
- `MetaFrom(r)` - 获取请求元数据
|
||||
|
||||
### 代码风格
|
||||
|
||||
1. **标准库优先**: 优先使用 Go 标准库，避免引入不必要的第三方依赖
|
||||
2. **包命名**: 使用小写单数形式，如 `api`, `config`, `db`
|
||||
3. **函数命名**:
|
||||
- Handler 函数使用 `XxxHandler` 命名
|
||||
- 中间件函数使用 `withXxx` 命名
|
||||
- 工具函数使用驼峰命名
|
||||
4. **错误处理**:
|
||||
- API 错误使用 `fail(w, r, status, msg)` 返回统一格式
|
||||
- 成功响应使用 `ok(w, r, data)` 返回统一格式
|
||||
- 启动错误使用 `log.Fatal()`
|
||||
5. **数据库操作**:
|
||||
- 使用 `database/sql` 标准库
|
||||
- 支持连接池配置（通过环境变量）
|
||||
- 多数据库实例：`metaDB` (模板/任务), `marketingDB` (营销数据), `ymtDB` (易码通数据)
|
||||
|
||||
### 响应格式
|
||||
|
||||
统一使用以下 JSON 响应结构：
|
||||
```go
|
||||
type resp struct {
|
||||
Code int `json:"code"` // 0 表示成功，非 0 表示错误
|
||||
Msg string `json:"msg"` // 消息
|
||||
Data interface{} `json:"data"` // 数据
|
||||
TraceID string `json:"trace_id"` // 追踪 ID
|
||||
}
|
||||
```
|
||||
|
||||
### 日志规范
|
||||
|
||||
1. **结构化日志**: 使用 `logging.JSON()` 记录结构化日志
|
||||
2. **访问日志**: 在 `withAccess` 中间件中自动记录，包含 method、path、status、duration 等
|
||||
3. **错误日志**: 在 `fail()` 函数中记录错误，包含 trace_id、文件位置、SQL（如果有）
|
||||
4. **日志格式**: JSON 格式，包含 level、ts、trace_id、method、path 等字段
|
||||
|
||||
## API 设计规范
|
||||
|
||||
### 路由规则
|
||||
|
||||
1. **RESTful 风格**:
|
||||
- `GET /api/templates` - 列表
|
||||
- `POST /api/templates` - 创建
|
||||
- `GET /api/templates/{id}` - 详情
|
||||
- `PATCH /api/templates/{id}` - 更新
|
||||
- `DELETE /api/templates/{id}` - 删除
|
||||
|
||||
2. **路径处理**: 使用 `strings.TrimPrefix()` 处理路径前缀，支持带或不带尾部斜杠
|
||||
|
||||
3. **CORS 支持**: 所有 API 通过 `withAccess` 中间件自动处理 CORS
|
||||
|
||||
### 请求处理
|
||||
|
||||
1. **参数解析**:
|
||||
- GET 请求从 `r.URL.Query()` 获取参数
|
||||
- POST 请求从 `r.Body` 读取 JSON
|
||||
- 使用 `json.Decoder` 或 `json.Unmarshal` 解析
|
||||
|
||||
2. **用户身份**: 通过 `userId` 查询参数传递（支持 `userId`, `userid`, `user_id`）
|
||||
|
||||
3. **权限检查**: 在 Handler 内部根据业务逻辑进行权限验证
|
||||
|
||||
## 数据库操作规范
|
||||
|
||||
### 连接管理
|
||||
|
||||
1. **多数据源**:
|
||||
- `metaDB`: 存储模板和任务元数据
|
||||
- `marketingDB`: 营销系统数据
|
||||
- `ymtDB`: 易码通数据
|
||||
|
||||
2. **连接池配置**: 通过环境变量配置（`YMT_DB_*`, `MARKETING_DB_*`, `YMT_TEST_DB_*`）
|
||||
|
||||
3. **DSN 格式**: 使用 `DB.DSN()` 方法生成，包含 `parseTime=True&loc=Local&charset=utf8mb4`
|
||||
|
||||
### Schema 抽象
|
||||
|
||||
1. **Schema 接口**: 使用 `schema.Schema` 接口抽象不同数据源的差异
|
||||
2. **字段映射**: 通过 `MapField()` 方法将逻辑字段名映射到物理字段名
|
||||
3. **表名映射**: 通过 `TableName()` 方法获取实际表名
|
||||
4. **JOIN 构建**: 通过 `BuildJoins()` 方法构建必要的 JOIN 语句
|
||||
|
||||
### SQL 构建
|
||||
|
||||
1. **SQL Builder**: 使用 `exporter.BuildSQL()` 构建查询 SQL
|
||||
2. **白名单机制**: 字段访问需要通过白名单验证
|
||||
3. **参数化查询**: 使用 `?` 占位符，防止 SQL 注入
|
||||
4. **时间过滤**: 营销系统的 order 表必须提供 `create_time_between` 过滤条件
|
||||
|
||||
## 配置管理规范
|
||||
|
||||
### 配置文件
|
||||
|
||||
1. **配置文件位置**:
|
||||
- 优先使用 `server/config.yaml`
|
||||
- 备选 `config.yaml`
|
||||
|
||||
2. **配置结构**:
|
||||
```yaml
|
||||
app:
|
||||
port: "8077"
|
||||
marketing_db:
|
||||
host: "..."
|
||||
port: "..."
|
||||
user: "..."
|
||||
password: "..."
|
||||
name: "..."
|
||||
ymt_db: {...}
|
||||
ymt_test_db: {...}
|
||||
ymt_key_decrypt_key_b64: "..."
|
||||
```
|
||||
|
||||
3. **环境变量覆盖**:
|
||||
- 支持通过环境变量覆盖配置
|
||||
- 环境变量优先级高于配置文件
|
||||
- 支持 `.env.local` 文件（不提交到版本控制）
|
||||
|
||||
## 前端代码规范
|
||||
|
||||
### Vue 3 使用
|
||||
|
||||
1. **CDN 引入**: 使用 CDN 方式引入 Vue 3 和 Element Plus，不打包
|
||||
2. **组合式 API**: 使用 `setup()` 和 `reactive()` 进行状态管理
|
||||
3. **API 调用**: 使用 `fetch()` 调用后端 API，base URL 为 `http://localhost:8077`
|
||||
4. **用户 ID**: 从 URL 查询参数获取 `userId`
|
||||
|
||||
### 代码组织
|
||||
|
||||
1. **单文件应用**: 主要逻辑在 `web/main.js` 中
|
||||
2. **样式分离**: 样式在 `web/styles.css` 中
|
||||
3. **静态资源**: 第三方库在 `web/vendor/` 目录
|
||||
|
||||
## 错误处理规范
|
||||
|
||||
1. **API 错误**:
|
||||
- HTTP 状态码：400 (Bad Request), 404 (Not Found), 500 (Internal Server Error)
|
||||
- 响应体：`{code: 1, msg: "错误消息", data: null, trace_id: "..."}`
|
||||
|
||||
2. **日志记录**:
|
||||
- 所有错误都记录到日志，包含 trace_id、文件位置、SQL（如果有）
|
||||
- 使用 `failCat()` 记录分类错误
|
||||
|
||||
3. **错误消息**: 使用中文错误消息，清晰描述问题
|
||||
|
||||
## 导出功能规范
|
||||
|
||||
1. **导出格式**: 支持 CSV 和 Excel (xlsx)
|
||||
2. **流式处理**: 使用 `exporter.Stream()` 进行流式导出，避免内存溢出
|
||||
3. **文件存储**: 导出文件存储在 `storage/export/` 目录
|
||||
4. **异步处理**: 导出任务异步执行，通过 job 状态跟踪进度
|
||||
|
||||
## 安全规范
|
||||
|
||||
1. **访问控制**: 通过 `withAccess` 中间件处理 CORS
|
||||
2. **SQL 注入防护**: 使用参数化查询，不拼接 SQL 字符串
|
||||
3. **字段白名单**: 字段访问必须通过白名单验证
|
||||
4. **敏感信息**: 密码等敏感信息不记录到日志
|
||||
|
||||
## 开发建议
|
||||
|
||||
1. **新增 API**:
|
||||
- 在 `internal/api/` 下创建新的 handler 文件
|
||||
- 在 `router.go` 中注册路由
|
||||
- 使用统一的响应格式和错误处理
|
||||
|
||||
2. **新增数据源**:
|
||||
- 在 `internal/schema/` 下实现 `Schema` 接口
|
||||
- 在 `schema.Get()` 中添加路由逻辑
|
||||
|
||||
3. **代码审查要点**:
|
||||
- 是否使用参数化查询
|
||||
- 是否通过字段白名单验证
|
||||
- 是否使用统一的响应格式
|
||||
- 是否记录必要的日志
|
||||
|
||||
4. **性能优化**:
|
||||
- 大数据量导出使用流式处理
|
||||
- 合理使用数据库连接池
|
||||
- 避免 N+1 查询问题
|
||||
|
||||
## 注意事项
|
||||
|
||||
1. **不要使用第三方 Web 框架**: 坚持使用标准库 `net/http`
|
||||
2. **保持代码简洁**: 避免过度抽象，保持代码可读性
|
||||
3. **统一错误处理**: 所有错误都通过 `fail()` 函数返回
|
||||
4. **日志完整性**: 确保关键操作都有日志记录
|
||||
5. **配置灵活性**: 支持配置文件和环境变量两种方式
|
||||
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
.git
|
||||
log/
|
||||
storage/
|
||||
server/bin/
|
||||
server/config.yaml
|
||||
server/config.test.yaml
|
||||
server/config.prod.yaml
|
||||
scripts/*.tar
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
server/log/
|
||||
storage/
|
||||
server/config.test.yaml
|
||||
server/config.prod.yaml
|
||||
server/config.yaml
|
||||
server/bin/
|
||||
scripts/*.tar
|
||||
|
|
@ -0,0 +1,651 @@
|
|||
# MarketingSystemDataTool 项目规则
|
||||
|
||||
## 1. 项目概述
|
||||
|
||||
- 技术栈：后端使用 `Go`，前端使用 `Vue 3 + Element Plus`（通过 CDN 引入，无打包）；前后端代码同仓管理。
|
||||
- 目标：为“营销系统”和“易码通系统”提供统一的高性能数据导出能力，支持模板化 SQL 构建与权限控制，生成 `CSV` 与 `Excel` 文件。
|
||||
- 数据源：至少包含两个独立库（`marketing_db`、`ymt_db`），通过环境变量配置连接与凭据。
|
||||
|
||||
## 2. 目录结构与命名约定
|
||||
|
||||
- `server/`：Go 服务端代码（API、模板校验、导出执行、权限校验、日志与监控）。
|
||||
- `web/`：前端页面与静态资源（模板管理、导出发起、进度查看、历史下载），采用 `Vue 3 + Element Plus` 组件搭建。
|
||||
- `config/`：非敏感配置（字段白名单、场景定义、关联关系元数据）。敏感信息使用环境变量注入。
|
||||
- `scripts/`：开发与运维脚本（如生成索引建议、批量校验 EXPLAIN）。
|
||||
- 命名规范：统一使用下划线或小驼峰，文件名见名知意；避免缩写导致歧义。
|
||||
|
||||
## 3. 权限与安全
|
||||
|
||||
- 权限字段：所有导出查询必须包含权限范围条件（如 `user_id`、`tenant_id`、`owner_id`），通过 `IN` 或等值过滤传入。
|
||||
- 白名单策略：仅允许查询白名单字段与表；禁止 `SELECT *` 与任意动态 SQL 拼接。
|
||||
- 参数化查询：所有条件、排序、分页使用预编译参数；禁止字符串拼接导致注入风险。
|
||||
- 敏感信息：数据库凭据、密钥等仅通过环境变量加载；严禁写入仓库与日志。
|
||||
- 审计日志：记录模板创建与变更、EXPLAIN 结果、导出执行参数、数据量、耗时、下载行为。
|
||||
|
||||
## 4. 导出场景定义
|
||||
|
||||
- 易码通系统：
|
||||
- 订单导出：订单类型包含 `直充卡密订单`、`立减金订单`、`红包订单`。
|
||||
- 支付记录导出：支付类型包含 `key码支付`、`商品支付`。
|
||||
- 营销系统：
|
||||
- 订单导出：订单类型包含 `直充卡密订单`、`立减金订单`、`红包订单`。
|
||||
- 主表定义：每个场景明确主表与主键、权限字段、必要索引；关联表通过元数据预定义关联键与可选字段。
|
||||
|
||||
## 5. SQL 模板规范
|
||||
|
||||
- 模板组成：
|
||||
- 选择数据源（库名）。
|
||||
- 选择数据类型（主表）。
|
||||
- 选择关联数据（关联表、连接类型、连接键）。
|
||||
- 字段选择（仅白名单字段，支持别名）。
|
||||
- 条件过滤（权限条件必填，业务条件可选）。
|
||||
- 排序与分页（必须与索引一致性，避免大偏移）。
|
||||
- 输出格式（`csv`/`xlsx`），列统计与多 sheet 选项。
|
||||
- 生成规则：
|
||||
- 自动生成 `JOIN` 与选择列，强制添加权限过滤。
|
||||
- 禁止笛卡尔积；必须给出明确连接条件。
|
||||
- 大表查询优先走索引覆盖；必要时使用 `CTE/子查询` 减少回表。
|
||||
- 统一走 `EXPLAIN` 评估；达标才允许执行。
|
||||
|
||||
## 6. EXPLAIN 评估标准
|
||||
|
||||
- 基本要求：
|
||||
- 禁止对大表 `type=ALL` 全表扫描；优先 `ref`/`range`/`const`。
|
||||
- `rows` 估算在权限范围内合理可控；避免千万级一次性拉取。
|
||||
- 正确使用连接顺序与驱动表；避免 `Using temporary`、`Using filesort` 发生在超大数据集。
|
||||
- 必要索引存在（权限字段、连接键、排序键、过滤键）。
|
||||
- 超标处理：
|
||||
- 自动生成索引建议（基于过滤列与排序列）。
|
||||
- 指导模板调整（减少字段、改用预聚合或分批）。
|
||||
|
||||
## 7. 导出执行与进度
|
||||
|
||||
- 执行模型：
|
||||
- 单次查询尽量覆盖需求，避免多次往返；必要时拆分批次流式写入。
|
||||
- 流式导出：服务端边拉取边写文件，控制内存与背压。
|
||||
- 批大小动态：根据 `rows` 估算与 I/O 速率自适应。
|
||||
- 进度追踪：
|
||||
- 记录总行数（估算与实际）、已写行数、当前批次、速率、剩余时间估算。
|
||||
- 状态：`queued`/`running`/`failed`/`completed`/`canceled`。
|
||||
- 历史下载与导出记录：模板、发起人、是否公共、时间窗口、文件摘要（行数/大小/格式）。
|
||||
|
||||
## 8. 文件生成与格式
|
||||
|
||||
- `csv`：
|
||||
- 统一 `UTF-8` 与 `\n`；首行写列名；转义逗号与引号。
|
||||
- 超大数据优先选择 `csv`，可并发分块写入后合并。
|
||||
- `xlsx`：
|
||||
- 单 sheet 行数受限（建议 ≤ 100 万）；超限自动分 sheet。
|
||||
- 多 sheet 划分：按指定条件（如订单类型、日期分区）生成；需稳定排序避免跨批次混入。
|
||||
- 列统计：
|
||||
- 支持 `count/sum/avg/min/max`；可生成统计 sheet 或尾部汇总。
|
||||
- 统计尽量在同一 SQL 通过窗口/聚合生成，避免二次扫描。
|
||||
|
||||
## 9. 前后端交互协议
|
||||
|
||||
- 模板管理 API：
|
||||
- `POST /api/templates` 创建模板并运行 `EXPLAIN` 校验。
|
||||
- `GET /api/templates` 列表/搜索；`GET /api/templates/{id}` 详情。
|
||||
- `POST /api/templates/{id}/validate` 重新校验与给出索引建议。
|
||||
- `PATCH /api/templates/{id}` 更新元数据（公共/个人、描述、可见范围）。
|
||||
- 导出执行 API：
|
||||
- `POST /api/exports` 发起导出（模板 + 权限范围 + 条件 + 格式 + 统计 + 多 sheet）。
|
||||
- `GET /api/exports/{id}` 进度与指标；`GET /api/exports/{id}/download` 下载文件。
|
||||
- `POST /api/exports/{id}/cancel` 取消任务。
|
||||
- 前端约定：
|
||||
- 使用 `Vue 3 + Element Plus`，通过 CDN 加载：`vue@3` 与 `element-plus`；页面在 `web/index.html` 中挂载。
|
||||
- 统一通过权限范围参数（如 `user_id IN (...)`）传递查询边界；由后端注入到 SQL。
|
||||
- 所有下拉/选择项来自白名单与元数据；禁止自由输入列名与表名；表单与日期选择使用 Element Plus 组件。
|
||||
|
||||
## 10. 性能与稳定性
|
||||
|
||||
- 索引策略：优先覆盖权限字段、关联键、排序键；定期审计慢查询。
|
||||
- 分批与限流：根据 `EXPLAIN` rows 估算选择批大小；对导出并发做全局限流。
|
||||
- 超时与重试：数据库查询、文件写入设置超时；幂等与断点续传支持。
|
||||
- 资源管理：严格控制连接池、内存占用、磁盘 I/O；避免阻塞主线程。
|
||||
|
||||
## 11. 测试与质量保障
|
||||
|
||||
- 单元测试：SQL 构建器、权限注入、EXPLAIN 评估、CSV/XLSX 写入器。
|
||||
- 集成测试：常见场景模板端到端导出与校验；超大数据模拟与性能门槛验证。
|
||||
- 基准测试：关键路径（查询与写文件）压测，建立性能基线与预算。
|
||||
|
||||
## 12. 日志、监控与告警
|
||||
|
||||
- 指标：导出次数、耗时、行数、失败率、平均速率、并发数。
|
||||
- 监控：数据库慢查询、队列堆积、磁盘空间、错误分布。
|
||||
- 告警：任务失败、耗时超阈值、EXPLAIN 超标、磁盘接近上限。
|
||||
|
||||
## 13. 版本控制与发布
|
||||
|
||||
- Git 工作流：特性分支 + 合并请求；语义化提交信息；模板变更需评审。
|
||||
- 发布：后端可编译为单二进制；前端静态资源打包；提供启动脚本与环境样例。
|
||||
- 变更管理：模板结构变更需迁移脚本与兼容策略；历史记录不丢失。
|
||||
|
||||
## 14. 角色与可见性
|
||||
|
||||
- 模板归属：支持公共与个人归属，公共模板需通过评审并标注责任人。
|
||||
- 访问控制：按角色/租户限制模板使用与导出权限；下载链接带有效期与签名。
|
||||
|
||||
## 15. 错误处理与用户体验
|
||||
|
||||
- 明确错误：权限缺失、EXPLAIN 未达标、索引建议、数据过大提示与替代方案。
|
||||
- 进度与估算：持续更新预计完成时间与当前速率；完成后通知。
|
||||
- 可恢复：失败任务可重试或继续；中断后支持断点续导出。
|
||||
|
||||
---
|
||||
|
||||
以上规则用于指导实现与评审，确保导出功能在安全、性能与可维护性方面达标。后续如有新场景与数据源接入，应当补充白名单与关联元数据，并通过测试与指标验证后上线。
|
||||
|
||||
## 16. 环境与连接配置
|
||||
|
||||
- 原则：所有连接信息通过环境变量注入，禁止将密码和密钥写入仓库或日志。
|
||||
- 营销系统关系型数据库（示例变量名）：
|
||||
- `MARKETING_DB_HOST`
|
||||
- `MARKETING_DB_PORT`
|
||||
- `MARKETING_DB_USER`
|
||||
- `MARKETING_DB_PASSWORD`
|
||||
- `MARKETING_DB_NAME`
|
||||
- `MARKETING_DB_MAX_OPEN_CONNS`、`MARKETING_DB_MAX_IDLE_CONNS`、`MARKETING_DB_CONN_MAX_LIFETIME`
|
||||
- 易码通系统关系型数据库（示例变量名）：
|
||||
- `YMT_DB_HOST`
|
||||
- `YMT_DB_PORT`
|
||||
- `YMT_DB_USER`
|
||||
- `YMT_DB_PASSWORD`
|
||||
- `YMT_DB_NAME`
|
||||
- 易码通系统 MySQL 驱动与 DSN：
|
||||
- 驱动：`mysql`（建议使用 `github.com/go-sql-driver/mysql`）。
|
||||
- DSN 组合（推荐在运行时通过环境变量组装）：
|
||||
- 模板：`<user>:<password>@tcp(<host>:<port>)/<dbname>?parseTime=True&loc=Local&charset=utf8mb4`
|
||||
- 对应变量：`YMT_DB_USER`、`YMT_DB_PASSWORD`、`YMT_DB_HOST`、`YMT_DB_PORT`、`YMT_DB_NAME`。
|
||||
- 连接选项：
|
||||
- `parseTime=True` 解析时间类型为 `time.Time`。
|
||||
- `loc=Local` 或设定为具体时区；如跨时区导出，优先统一到 `UTC` 并在前端展示时转换。
|
||||
- `charset=utf8mb4` 以避免字符集问题。
|
||||
- 生产建议：启用只读账号、限定白名单 IP、必要时开启 TLS，禁止在仓库或日志中存储明文密码与完整 DSN。
|
||||
- 营销系统 Redis（缓存/队列等，示例变量名）：
|
||||
- `MARKETING_REDIS_HOST`
|
||||
- `MARKETING_REDIS_PORT`
|
||||
- `MARKETING_REDIS_PASSWORD`
|
||||
- 使用规范：
|
||||
- 本地与生产环境分别通过系统环境或安全的配置服务注入上述变量。
|
||||
- 连接池、超时与重试策略必须可配置：查询/执行超时、读写超时、最大重试次数与退避策略。
|
||||
- 优先开启 TLS/SSL（如数据库与 Redis 支持），并使用白名单 IP 与最小权限账号。
|
||||
- 在运行与监控面板中仅显示掩码信息（如 `***`），避免泄露。
|
||||
|
||||
## 17. 易码通数据库与表设计
|
||||
|
||||
- 存储位置：所有导出相关业务数据统一存储在易码通 MySQL（库：`merketing`），使用 `InnoDB` 与 `utf8mb4`，时间统一为 `UTC`（前端展示时转换）。
|
||||
- 表清单与职责：
|
||||
- `export_templates`（导出模板主表）：
|
||||
- 字段：`id`、`name`、`datasource`（`ymt`/`marketing`）、`main_table`、`joins_json`、`fields_json`、`filters_json`、`file_format`（`csv`/`xlsx`）、`stats_enabled`、`sheet_split_by`、`visibility`（`public`/`private`）、`owner_id`、`enabled`、`explain_json`、`explain_score`、`last_validated_at`、`created_at`、`updated_at`。
|
||||
- 索引：`owner_id`、`visibility`、`enabled`、`created_at`。
|
||||
- `export_jobs`（导出任务表）：
|
||||
- 字段：`id`、`template_id`、`status`（`queued`/`running`/`failed`/`completed`/`canceled`）、`requested_by`、`permission_scope_json`（如 `user_ids`、`tenant_id` 范围）、`options_json`、`row_estimate`、`total_rows`、`file_format`、`started_at`、`finished_at`、`created_at`、`updated_at`。
|
||||
- 索引：`template_id`、`status`、`requested_by`、`created_at`。
|
||||
- `export_job_files`（导出文件记录）：
|
||||
- 字段：`id`、`job_id`、`storage_uri`、`sheet_name`、`row_count`、`size_bytes`、`checksum`、`created_at`。
|
||||
- 索引：`job_id`。
|
||||
- `export_audits`（审计与变更记录）：
|
||||
- 字段：`id`、`actor_id`、`action`、`entity_type`、`entity_id`、`detail_json`、`created_at`。
|
||||
- 索引：`entity_type+entity_id`、`actor_id`、`created_at`。
|
||||
- 可选：`export_metrics_daily`（按日聚合指标）：`date`、`template_id`、`jobs_count`、`rows_count`、`avg_duration_ms` 等。
|
||||
- 约束与规范：
|
||||
- 外键：`export_jobs.template_id → export_templates.id`，`export_job_files.job_id → export_jobs.id`；模板被引用时 `ON DELETE RESTRICT`，避免误删。
|
||||
- 主键：统一 `BIGINT UNSIGNED` 自增；布尔使用 `TINYINT(1)`；JSON 字段用于结构化模板与权限范围。
|
||||
- 命名：`snake_case`；时间字段包含 `created_at`、`updated_at`，应用层维护更新时间。
|
||||
- 权限与多租户：
|
||||
- 表层面包含 `owner_id` 或 `tenant_id` 以便分区过滤；任务执行范围记录于 `permission_scope_json`（如 `user_ids`、`tenant_id`、`date_range`）。
|
||||
- 所有查询必须依赖权限字段建索引并注入过滤条件。
|
||||
- 索引策略：
|
||||
- 组合索引：`status, created_at`、`owner_id, created_at`、`template_id, created_at`，满足列表与检索高频路径。
|
||||
- 针对主表与关联键建立覆盖索引，降低回表与避免 `Using filesort/temporary`。
|
||||
- 迁移与版本：
|
||||
- 在 `server/migrations` 维护迁移脚本（`up/down`），每次结构变更必须提供兼容策略与数据迁移方案。
|
||||
- 变更需通过测试与评审，记录在审计表中并与发布流程绑定。
|
||||
- 数据保留与清理：
|
||||
- 历史导出文件仅保留元数据与存储 URI，按策略定期清理物理文件；失败/取消任务保留审计但可清理文件记录。
|
||||
- 性能与安全：
|
||||
- 仅存储模板与任务元数据，不冗余导出结果数据；避免在表中存储敏感凭据；对大查询任务做限流与队列管理。
|
||||
|
||||
## 18. 营销系统订单主表定义
|
||||
|
||||
- 主表：`order`（保留字，SQL 中需使用反引号包裹：`` `order` ``）。
|
||||
- 主键：`order_number`（`VARCHAR(20)`）。
|
||||
- 权限字段：`creator`（`INT UNSIGNED`），用于用户权限过滤与范围查询，前端导出通过 `creator IN (...)` 传入范围参数。
|
||||
- 字段要点：
|
||||
- `` `key` ``（保留字，需使用反引号）：业务 KEY。
|
||||
- `out_trade_no`：支付流水号。
|
||||
- `type`：订单类型，建议映射：`1=直充卡密`、`2=立减金`、`3=红包`（如与现网不一致，以现网为准）。
|
||||
- `status`：订单状态；`deliver_status`：向上游投递状态。
|
||||
- `account`、`product_id`、`reseller_id`、`plan_id`、`key_batch_id`、`code_batch_id`。
|
||||
- `contract_price`、`num`、`total`（生成列：`contract_price * num`）、`pay_amount`、`pay_type`、`pay_status`、`use_coupon`。
|
||||
- `expire_time`、`recharge_time`、`create_time`、`update_time`。
|
||||
- `card_code`：敏感信息字段，默认不允许导出；需更高权限并进行掩码或脱敏处理。
|
||||
- 索引：
|
||||
- `PRIMARY KEY (order_number)`。
|
||||
- `idx_out_trade_no (out_trade_no)`、`idx_key (` `key` `)`、`idx_code_batch_id (code_batch_id)`、`idx_product_id (product_id)`、`idx_reseller_id (reseller_id)`、`idx_create_time_account (account, create_time)`。
|
||||
- 复合索引：`idx_create_time_creator (create_time, creator)`、`idx_plan_id_create_time_creator (plan_id, create_time, creator)`。
|
||||
- 导出查询规范：
|
||||
- 所有营销系统订单导出必须包含 `creator` 范围过滤（例如：`creator IN (?)` 或等值），并优先结合 `create_time` 使用复合索引。
|
||||
- 订单类型筛选通过 `type IN (1,2,3)`；支付相关通过 `pay_status`、`out_trade_no`；避免对未建索引列进行大范围过滤。
|
||||
- 严禁 `SELECT *`；仅允许白名单列（默认排除 `card_code`）。
|
||||
- 时间范围必填，建议按 `create_time` 进行分区或限窗，避免跨超大时间窗口的全量拉取。
|
||||
- EXPLAIN 达标建议：
|
||||
- 驱动表选择 `order`，过滤条件包含 `creator` 与 `create_time`；`type`、`plan_id`、`reseller_id` 作为附加过滤。
|
||||
- 避免 `Using filesort/temporary` 在百万级数据上出现；如需排序，尽量使用已存在的复合索引顺序。
|
||||
- 安全与合规：
|
||||
- 对 `card_code` 输出采用掩码（如仅展示前 6 后 4）；权限不足禁止选取该列。
|
||||
- 对支付流水号与账号字段进行最小必要输出；下载文件带签名与有效期。
|
||||
- 命名注意：
|
||||
- 表名 `order` 与列名 `` `key` ``为保留字，所有 SQL 需使用反引号包裹以避免语法冲突。
|
||||
|
||||
## 19. 营销系统订单详情表定义
|
||||
|
||||
- 表：`order_detail`（与主表 `order` 通过 `order_number` 一一关联）。
|
||||
- 主键：`order_number`（`VARCHAR(20)`）。
|
||||
- 关联关系：`order_detail.order_number = order.order_number`；查询时以 `order` 为驱动表进行 `INNER JOIN` 或 `LEFT JOIN`。
|
||||
- 字段要点：
|
||||
- `plan_title`、`reseller_name`、`product_name`：冗余展示字段，来自计划/分销商/商品。
|
||||
- `show_url`：商品图片 URL（冗余）。
|
||||
- `official_price`、`cost_price`：官方价与成本价；与 `order.contract_price` 有区分。
|
||||
- `product`：冗余商品信息（`JSON`）。
|
||||
- `refund_account`：退款打款账号（敏感，默认不导出或需掩码）。
|
||||
- `create_time`、`update_time`：时间字段，跟随主表时区与展示规范。
|
||||
- 索引：
|
||||
- `PRIMARY KEY (order_number)`。
|
||||
- 建议增加：`idx_update_time (update_time)` 或 `idx_create_time (create_time)` 视查询需要；如按时间窗口筛选详情。
|
||||
- 导出查询规范：
|
||||
- 所有详情导出需通过与主表 `order` 关联，并在主表上应用权限过滤：`creator IN (...)` 与时间范围。
|
||||
- 字段白名单：默认允许 `plan_title/reseller_name/product_name/show_url/official_price/cost_price/create_time/update_time`；`refund_account` 与 `product`（JSON）需更高权限或进行脱敏/选择性字段输出。
|
||||
- 避免单独对 `order_detail` 进行大范围扫描；以主表为驱动确保使用主表的复合索引。
|
||||
- EXPLAIN 达标建议：
|
||||
- 连接以主表为驱动（主表过滤 `creator + create_time`），详情表通过主键或索引快速匹配，避免 `Using temporary` 与大范围 `filesort`。
|
||||
- 如需按详情时间排序，确保存在相应时间索引或采用分页与稳定排序键。
|
||||
- 安全与合规：
|
||||
- 对 `refund_account` 默认隐藏或掩码（如仅展示部分）；`product` JSON 仅输出必要键，避免泄露多余属性。
|
||||
- 下载与展示遵循最小必要原则，确保敏感字段在模板白名单中默认关闭。
|
||||
|
||||
## 20. 营销系统红包订单副表定义
|
||||
|
||||
- 表：`order_cash`（与主表 `order` 通过 `order_number` 一一关联）。
|
||||
- 主键：`order_number`（`VARCHAR(20)`）。
|
||||
- 关联关系：`order_cash.order_number = order.order_number`；查询以主表为驱动表进行 `INNER JOIN` 或 `LEFT JOIN`。
|
||||
- 字段要点：
|
||||
- `channel`：渠道（`1=支付宝`、`2=微信`）。
|
||||
- `cash_activity_id`：红包批次号；`cash_packet_id`：红包 ID；`cash_id`：红包模板/规则 ID。
|
||||
- `receive_status`：领取状态；`receive_time`：拆红包时间。
|
||||
- `channel_order_id`：支付宝转账订单号/微信批次单号；`pay_fund_order_id`：支付宝资金流水；`wechat_detail_id`：微信商家明细单号。
|
||||
- `receive_user_id`：领取者唯一标识（微信 `open_id`/支付宝 `alipay_user_id`），属于敏感标识，默认不导出或需脱敏。
|
||||
- `amount`：红包额度；`status`：状态（`1正常`、`2过期`）；`expire_time`：过期时间。
|
||||
- `receive_name`：领取方真实姓名（PII，默认不导出或掩码）。
|
||||
- `update_time`：更新时间（可用于按更新时间筛选）。
|
||||
- 索引：
|
||||
- `PRIMARY KEY (order_number)` 与 `UNIQUE o_uidx (order_number)`（二者功能重合，建议保留一个唯一约束以避免重复约束）。
|
||||
- `r_time_idx (receive_time)`、`r_c_idx (receive_user_id, cash_id)`、`idx_expire_time_status (expire_time, status)`。
|
||||
- 导出查询规范：
|
||||
- 红包订单导出需与主表 `order` 关联，并在主表上应用权限过滤：`creator IN (...)` 和时间范围。
|
||||
- 字段白名单：默认允许 `channel/cash_activity_id/receive_status/receive_time/cash_packet_id/cash_id/amount/status/expire_time/update_time`；`receive_user_id/receive_name/channel_order_id/pay_fund_order_id/wechat_detail_id` 需更高权限或掩码处理。
|
||||
- 避免对未索引字段进行大范围过滤；按 `receive_time` 或 `expire_time,status` 进行筛选时使用相应索引。
|
||||
- EXPLAIN 达标建议：
|
||||
- 以主表为驱动（主表过滤 `creator + create_time`），副表通过主键或索引匹配；避免在大数据集上出现 `Using temporary` 与 `filesort`。
|
||||
- 排序尽量使用已存在索引的前缀；对 `receive_time` 排序应结合筛选与分页。
|
||||
- 安全与合规：
|
||||
- `receive_user_id` 与 `receive_name` 属 PII，模板默认关闭并需权限校验与脱敏（如散列或局部显示）。
|
||||
- 渠道订单/资金流水号仅在合规范围内输出，避免泄露与被用作外部关联键。
|
||||
|
||||
## 21. 营销系统立减金订单表定义
|
||||
|
||||
- 表：`order_voucher`（与主表 `order` 通过 `order_number` 关联；亦可通过唯一 `trade_no` 与外部交易做关联）。
|
||||
- 主键：`id`（自增，`INT`）。唯一键：`trade_no`。
|
||||
- 关联关系：`order_voucher.order_number = order.order_number`；查询以主表为驱动表进行 `INNER JOIN` 或 `LEFT JOIN`。
|
||||
- 字段要点：
|
||||
- 渠道与批次：`channel`（`1=支付宝`、`2=微信`）、`channel_activity_id`（渠道立减金批次）、`channel_voucher_id`（渠道立减金 ID）。
|
||||
- 用户标识：`channel_user_id`（渠道用户 ID，PII，默认不导出或需脱敏）。
|
||||
- 规则与状态：`rule`（立减金规则，`JSON`）、`status`（`1可用/2已实扣/3已过期/4已退款`）。
|
||||
- 时间：`grant_time`（领取）、`usage_time`（核销）、`refund_time`（退款）、`status_modify_time`（状态更新时间）、`overdue_time`（过期）。
|
||||
- 价格与账户：`official_price`、`refund_amount`、`receive_mode`（`1渠道授权用户id/2手机号或邮箱`）、`receive_error`、`app_id`、`out_biz_no`、`fail_time`、`notify_url`、`account_no`。
|
||||
- 索引：
|
||||
- `PRIMARY KEY (id)`、`UNIQUE idx_trade_no (trade_no)`。
|
||||
- `idx_order_number (order_number)`、`idx_channel_uid (channel, channel_user_id)`、`idx_channel_user_id (channel_user_id, channel)`、`idx_channel_voucher_id (channel_voucher_id)`、`idx_activity_voucher (channel_activity_id, channel)`、`index_status (status)`、`idx_usage_time (usage_time)`、`idx_out_biz_no (out_biz_no)`。
|
||||
- 导出查询规范：
|
||||
- 立减金订单导出需与主表 `order` 关联，并在主表上应用权限过滤：`creator IN (...)` 与时间范围；避免对本表进行独立全表扫描。
|
||||
- 字段白名单：默认允许 `channel/channel_activity_id/channel_voucher_id/status/grant_time/usage_time/refund_time/status_modify_time/overdue_time/refund_amount/official_price/out_biz_no/account_no`；`channel_user_id/app_id/notify_url/receive_error` 与 `rule`（JSON）需更高权限或脱敏后选择性输出。
|
||||
- 规则字段 `rule` 建议由模板定义具体键路径进行选择性导出，避免整段 JSON 输出造成泄露与体积膨胀。
|
||||
- EXPLAIN 达标建议：
|
||||
- 以主表为驱动（主表过滤 `creator + create_time`），本表通过 `order_number` 或其他索引列匹配；对时间或状态筛选使用对应索引。
|
||||
- 如需要按 `usage_time` 或 `status` 排序与筛选，确保选择性足够并配合分页，避免 `Using filesort/temporary` 在大数据集上出现。
|
||||
- 安全与合规：
|
||||
- `channel_user_id` 属 PII，模板默认关闭并需脱敏（如散列或局部显示）；`account_no` 亦需最小必要输出与掩码。
|
||||
- `notify_url/out_biz_no/app_id` 等用于渠道交互的字段不用于外部分发，谨慎输出并记录审计。
|
||||
|
||||
## 22. 营销系统活动计划表定义
|
||||
|
||||
- 表：`plan`（营销计划与活动配置）。
|
||||
- 主键与唯一：`id`（自增，`INT UNSIGNED`）；唯一键 `stock_id`（计划编号/批次号）。
|
||||
- 关联关系：`order.plan_id = plan.id`；订单侧查询通过主表驱动与计划表关联。
|
||||
- 权限字段：`creator`（数据所属用户）；可补充按 `reseller_id` 进行租户/渠道范围过滤。
|
||||
- 字段要点：
|
||||
- 创建者：`creator`、`creator_name`。
|
||||
- 基本信息：`title`（计划标题）、`type`（计划类型）、`status`（活动状态，`0草稿` 等）。
|
||||
- 分销商：`reseller_id`、`reseller_name`（冗余）。
|
||||
- 时间：`begin_time`、`end_time`（活动窗口）、`create_time`、`update_time`、`delete_time`（软删除）。
|
||||
- 计划与结算：`return_type`、`open`（1开启/2关闭）、`settlement_type`、`send_method`（1邮件/2API）。
|
||||
- 压缩包与安全：`zip_file`、`zip_file_md5`、`zip_pwd`（敏感，默认不导出或需脱敏/掩码）。
|
||||
- 其他：`approval_id`、`copy_count`、`stock_id`、`button_conf`（`JSON`）。
|
||||
- 索引：
|
||||
- `udx_stock_id (stock_id)` 唯一；`idx_reseller (begin_time, end_time, reseller_id)`；`idx_status_creator (status, creator)`；`idx_end_time (end_time)`。
|
||||
- 建议：如存在常用 `begin_time` 范围筛选与按状态查询，可添加 `status, begin_time` 复合索引以提高选择性。
|
||||
- 导出查询规范：
|
||||
- 计划相关导出需结合主表订单或自身权限字段进行过滤：`creator IN (...)` 或 `reseller_id IN (...)` 与时间窗口 `begin_time ≤ t ≤ end_time`。
|
||||
- 默认白名单不包含 `zip_pwd/zip_file_md5` 等敏感字段；如需导出，必须具备更高权限并进行掩码或仅输出校验摘要。
|
||||
- 对软删除记录：导出与展示需排除 `delete_time IS NOT NULL` 条目（或由模板显式选择策略）。
|
||||
- EXPLAIN 达标建议：
|
||||
- 针对计划列表与关联查询，优先使用 `status, creator` 与 `begin_time, end_time, reseller_id` 组合索引；时间窗口筛选需与索引前缀一致以避免全表扫描。
|
||||
- 计划与订单关联时，以订单为驱动（带 `creator + create_time` 的过滤），计划表通过主键或索引快速匹配，避免 `Using temporary/filesort`。
|
||||
- 安全与合规：
|
||||
- `zip_pwd` 等敏感信息严格限制输出并进行审计；`button_conf`（JSON）默认关闭或选择性键导出。
|
||||
- 下载与展示遵循最小必要原则，模板白名单默认不包含敏感字段；所有导出操作记录审计。
|
||||
|
||||
## 23. 营销系统 KEY 批次表定义
|
||||
|
||||
- 表：`key_batch`（KEY 批次与库存配置）。
|
||||
- 主键：`id`（自增，`INT UNSIGNED`）。
|
||||
- 关联关系：`key_batch.plan_id = plan.id`；订单侧冗余引用为 `order.key_batch_id`。
|
||||
- 权限字段与范围：
|
||||
- `creator`（创建人用户 ID）用于权限过滤与范围查询；与计划表 `creator` 协同过滤。
|
||||
- 结合 `plan_id` 与活动时间窗口进行边界限定（`begin_time ≤ t ≤ end_time`）。
|
||||
- 字段要点：
|
||||
- 基本信息：`style`（KEY 样式）、`batch_name`（批次名称）。
|
||||
- 绑定与数量：`bind_object`（1兑换码/2优惠码/4立减金）、`quantity`（发放数量）、`stock`（剩余库存）、`restrict`（最大绑定数量）。
|
||||
- 行为配置：`allow_repetition`（是否允许重复选商品）、`merge_stock`（是否合并库存）、`allow_loss`（是否允许亏损）。
|
||||
- 时间与状态：`begin_time`、`end_time`、`status`、`create_time`、`update_time`、`delete_time`（软删除）、`discard_time`（作废时间）。
|
||||
- 冗余与敏感：`code_batch`（关联兑换码批次 JSON）、`zip_file/zip_file_md5/zip_pwd`（敏感，默认不导出或需脱敏/掩码）。
|
||||
- 成本与价格：`key_official_price`、`key_cost_price`；有效期与预警：`expiration_conf`、`warning_conf`（JSON）。
|
||||
- 其他：`generate_id`（生成记录 ID）、`creator_name`、`approval_status`、`approval_id`、`mobile_excel`（白名单 Excel 路径）、`mobile_repeat`（白名单允许重复号码 JSON）、`copy_count`。
|
||||
- 索引：
|
||||
- 现有：`idx_end_time (end_time)`、`idx_plan_id (plan_id)`。
|
||||
- 建议：如常按 `status + end_time` 或 `begin_time + end_time` 过滤，增加相应复合索引；对 `creator + create_time` 的审计或列表查询可新增复合索引以提高选择性。
|
||||
- 导出查询规范：
|
||||
- 所有与 KEY 批次相关的导出需结合权限：`creator IN (...)` 或通过计划与订单的权限边界进行过滤；必须包含时间窗口限定。
|
||||
- 字段白名单：默认允许 `plan_id/style/batch_name/bind_object/quantity/stock/allow_repetition/merge_stock/allow_loss/begin_time/end_time/status/create_time/update_time/key_official_price/key_cost_price`；敏感字段 `zip_file/zip_file_md5/zip_pwd/mobile_excel/mobile_repeat/code_batch/expiration_conf/warning_conf` 默认关闭或仅选择性键导出。
|
||||
- 禁止对未建索引字段进行大范围过滤；尽量以计划或订单为驱动进行关联查询，减少在批次表上的独立全表扫描。
|
||||
- EXPLAIN 达标建议：
|
||||
- 过滤条件包含 `plan_id` 与时间窗口；如按 `status` 或 `creator` 列表查询，需配合复合索引并分页，避免 `Using filesort/temporary`。
|
||||
- 与订单或计划表关联时，以高选择性过滤的表作驱动（通常订单或计划），批次表通过主键/索引快速匹配。
|
||||
- 安全与合规：
|
||||
- 任何涉及白名单文件路径、压缩包与密码的字段默认不导出；需要更高权限且输出内容需进行掩码或摘要化（如仅输出 MD5 校验）。
|
||||
- JSON 字段导出走模板键路径选择，避免整段输出造成泄露与体积膨胀；所有导出操作记录审计。
|
||||
|
||||
## 24. 营销系统兑换码批次表定义
|
||||
|
||||
- 表：`code_batch`（兑换码/优惠券批次信息）。
|
||||
- 主键：`id`（自增，`INT UNSIGNED`）。
|
||||
- 关联关系：
|
||||
- `code_batch.key_batch_id = key_batch.id`（外键逻辑关联）。
|
||||
- 订单侧冗余引用为 `order.code_batch_id`，查询时以订单或 KEY 批次为驱动进行关联。
|
||||
- 权限字段：`creator`（创建者用户 ID），用于导出权限过滤与范围控制。
|
||||
- 字段要点：
|
||||
- 基本：`status`、`title`、`plan_title`（冗余）、`describe`（兑换说明）、`range`（使用范围描述）。
|
||||
- 时间：`begin_time`、`end_time`、`create_time`、`update_time`、`delete_time`、`discard_time`。
|
||||
- 数量与库存：`quantity`、`usage`、`invalid`、`stock`（生成列：`(quantity - usage) - invalid`）。
|
||||
- 限额与类型：`restrict`（绑定限额次数）、`type`（`0兑换码/1优惠券`）、`recharge_type`（`1单个充值/2组合充值`）。
|
||||
- 价格与组合：`full`（满额）、`reduce`（减额）、`group_info`（组合商品基础信息，`JSON`）。
|
||||
- 审批与复制：`approval_status`、`approval_id`、`copy_count`。
|
||||
- 产品冗余：`product`（`JSON`）。
|
||||
- 周期配置：`period_type`（`1不设置/2自动/3手动`）、`period_num`、`period_day`、`period_fixed_receive_time`（`time`）。
|
||||
- 索引：
|
||||
- 现有：`key_batch_id (key_batch_id)`、`idx_end_time (end_time)`。
|
||||
- 建议：常用筛选可增加 `creator, create_time` 或 `status, end_time` 复合索引以提升选择性；按标题/计划模糊查询建议走前端限定与分页，避免无索引扫描。
|
||||
- 导出查询规范：
|
||||
- 导出需在主表（订单或 KEY 批次）应用权限与时间过滤：`creator IN (...)`、`begin_time ≤ t ≤ end_time`，再关联到 `code_batch`。
|
||||
- 字段白名单：默认允许 `title/status/begin_time/end_time/quantity/usage/invalid/stock/restrict/type/recharge_type/full/reduce/create_time/update_time`；`product/group_info/range`（大字段/JSON）默认关闭，需更高权限且建议选择性键导出。
|
||||
- 大范围筛选避免对未索引列进行；按 `end_time` 或 `status` 过滤时利用对应索引并配合分页。
|
||||
- EXPLAIN 达标建议：
|
||||
- 以订单或 KEY 批次为驱动（带 `creator + 时间窗口` 过滤），`code_batch` 通过主键或索引列快速匹配；避免 `Using temporary/filesort` 在大数据集出现。
|
||||
- 对组合/周期场景，必要时拆分查询或利用窗口函数在可支持的引擎上完成统计，减少回表与二次扫描。
|
||||
- 安全与合规：
|
||||
- `product`、`group_info`、`range` 等可能包含敏感或大体积信息，模板默认不导出；若需要输出，采用键路径与掩码策略并记录审计。
|
||||
- 软删除记录（`delete_time IS NOT NULL`）默认不参与导出，除非模板明确要求。
|
||||
|
||||
## 25. 营销系统 key 码发放记录表定义
|
||||
|
||||
- 表：`merchant_key_send`（开放平台 key 码发放记录）。
|
||||
- 主键与唯一：
|
||||
- `PRIMARY KEY (id)`（`BIGINT UNSIGNED` 自增）。
|
||||
- `UNIQUE udx_key (key)`（单码唯一）。
|
||||
- `UNIQUE udx_merchant_id_out_biz_no (merchant_id, out_biz_no)`（商户侧业务号唯一）。
|
||||
- 关联关系：
|
||||
- `key_batch_id → key_batch.id`；`plan_id → plan.id`；`stock_id` 与 `plan.stock_id` 逻辑关联。
|
||||
- 可与订单主表通过 `` `order`.`key` = merchant_key_send.key `` 进行关联以查询核销状态与订单信息。
|
||||
- 权限与范围：
|
||||
- 按 `reseller_id`（分销商）或 `merchant_id`（开放商户）进行范围过滤；前端导出通过传递 `reseller_id IN (...)` 或 `merchant_id IN (...)`。
|
||||
- 必须包含时间窗口（如 `create_time` 或 `status_update_time`）与状态筛选以提高选择性。
|
||||
- 字段要点：
|
||||
- 基本：`reseller_id`、`merchant_id`、`out_biz_no`、`out_timestamp`、`key_batch_id`、`stock_id`、`plan_id`、`store_id`。
|
||||
- 券码：`key`、`status`（`1待发放/2已核销/3已作废/4充值中`）、`num`（发放数量）。
|
||||
- 客户端与账号：`attach`（客户端请求参数，`JSON`）、`account_type`、`account`、`send_msg`（短信是否发送）。
|
||||
- 时间：`usage_time`（核销时间）、`discard_time`（作废时间）、`status_update_time`（状态变更时间）、`update_time`、`create_time`。
|
||||
- 索引：
|
||||
- 现有：`udx_key (key)`、`udx_merchant_id_out_biz_no (merchant_id, out_biz_no)`、`idx_merchant_id_stock_id (merchant_id, stock_id)`、`idx_stock_id (stock_id)`、`idx_out_biz_no (out_biz_no)`、`idx_create_time (create_time)`。
|
||||
- 建议：如常按状态或时间查询，增加 `status, status_update_time` 复合索引；针对分销商或商户维度增加 `reseller_id, create_time` 或 `merchant_id, create_time` 复合索引。
|
||||
- 导出查询规范：
|
||||
- 作为主导出场景时，必须包含权限范围（`reseller_id/merchant_id`）与时间窗口（`create_time/status_update_time`）；必要条件如 `status IN (...)`。
|
||||
- 与订单或批次关联导出时，以高选择性表为驱动（如订单带 `creator + create_time` 过滤 或 批次/计划带时间与状态过滤）。
|
||||
- 字段白名单：默认允许 `reseller_id/merchant_id/out_biz_no/out_timestamp/key_batch_id/stock_id/plan_id/store_id/key/status/num/usage_time/discard_time/status_update_time/update_time/create_time`；`attach/account` 等敏感或 PII 默认关闭或掩码输出（如部分显示或哈希）。
|
||||
- EXPLAIN 达标建议：
|
||||
- 过滤优先使用 `merchant_id/reseller_id + create_time` 或 `status + status_update_time` 组合索引；对 `key` 的点查走唯一索引。
|
||||
- 排序尽量与索引前缀一致并配合分页，避免在大数据集上 `Using filesort/temporary`。
|
||||
- 安全与合规：
|
||||
- `account` 属 PII，模板默认不导出；如需导出，必须具备更高权限与掩码策略。
|
||||
- `attach` JSON 仅导出必要键；严禁导出包含凭据或签名的字段；所有导出操作记录审计。
|
||||
|
||||
## 26. 营销系统立减金表定义
|
||||
|
||||
- 表：`voucher`（立减金批次与预算配置）。
|
||||
- 主键与唯一：`id`（自增，`INT UNSIGNED`）；唯一键 `channel_activity_id`（渠道立减金批次号）。
|
||||
- 关联关系：
|
||||
- 与立减金订单表 `order_voucher.channel_activity_id = voucher.channel_activity_id` 进行关联；
|
||||
- 可与商品 `goods_id` 关联到商品表（如存在），与订单的 `product_id` 做间接关联。
|
||||
- 字段要点：
|
||||
- 渠道与批次：`channel`（`1支付宝/2微信`）、`channel_activity_id`、`batch_goods_name`。
|
||||
- 金额与额度：`price`（合同单价）、`recharge_amount`（充值批次金额）、`frozen_amount`（冻结额度）、`balance`（剩余额度）、`used_amount`（生成列：`recharge_amount - balance`）、`denomination`（面额）、`reduce_amount`（立减额）。
|
||||
- 预算：`all_budget`（总预算）、`day_budget`（单天预算）。
|
||||
- 配置：`receive_conf`（领取配置，`JSON`）、`card_type`（卡种类型，`JSON`）、`time_limit`（时间限制配置，`JSON`）。
|
||||
- 说明与通知：`instruction`（使用说明，`TEXT`）、`early_per`（预警百分比，`JSON`）、`early_notifier`（预警通知人，`JSON`）、`last_early_per`（上次预警百分比）。
|
||||
- 模板与渠道：`temp_no`（券模板编号，仅支付宝）、`provider`（微信服务商标识，可选）。
|
||||
- 发券方式与数量：`receive_mode`（`1渠道用户id/2手机号或邮箱`）、`send_num`（发放数量）、`is_webview`（是否 webview 方式）。
|
||||
- 其他：`notice`（使用须知）、`index`（排序）、`goods_id`（商品）。
|
||||
- 时间：`create_time`、`delete_time`（软删除）。
|
||||
- 索引：
|
||||
- 现有：唯一 `channel_activity_id`，`idx_goods_id (goods_id)`。
|
||||
- 建议：如常按渠道与批次过滤，增加 `channel, channel_activity_id` 复合索引（若不会与唯一冲突的查询场景）；按创建时间列表查询可添加 `create_time` 索引；若按状态类筛选（来自配置），考虑在业务层限定，避免无选择性扫描。
|
||||
- 导出查询规范：
|
||||
- 立减金导出应通过与 `order_voucher` 或订单主表关联来继承权限边界（如 `creator IN (...)` 或租户范围），避免对 `voucher` 进行独立全表导出。
|
||||
- 字段白名单：默认允许 `channel/channel_activity_id/batch_goods_name/price/recharge_amount/frozen_amount/balance/used_amount/denomination/reduce_amount/all_budget/day_budget/provider/receive_mode/send_num/is_webview/create_time`；大字段/JSON（`receive_conf/card_type/time_limit/early_per/early_notifier/instruction/notice`）默认关闭或选择性键导出。
|
||||
- 如需按商品维度导出，需通过授权的 `goods_id IN (...)` 列表或与订单/商品白名单关联实现权限控制。
|
||||
- EXPLAIN 达标建议：
|
||||
- 关联查询以 `order_voucher` 或订单为驱动（带 `creator + 时间窗口` 过滤），`voucher` 通过唯一或索引列快速匹配；
|
||||
- 列表查询按 `create_time` 或 `channel_activity_id` 做点查/前缀匹配并分页，避免 `Using filesort/temporary`。
|
||||
- 安全与合规：
|
||||
- JSON 与说明类大字段默认不导出；若需输出，采用键路径与最小必要原则，并记录审计。
|
||||
- 避免导出可能包含渠道敏感配置或内部标识（如 `temp_no/provider`）到公共模板；需权限校验与用途限定。
|
||||
|
||||
## 27. 营销系统立减金批次表定义
|
||||
|
||||
- 表：`voucher_batch`（立减金批次与渠道映射）。
|
||||
- 主键与唯一：`id`（自增，`INT UNSIGNED`）；唯一键 `udx_channel_activity_id (channel_activity_id)`。
|
||||
- 关联关系：
|
||||
- `voucher_batch.voucher_id = voucher.id`（立减金主表关联）。
|
||||
- 与立减金订单 `order_voucher.channel_activity_id = voucher_batch.channel_activity_id` 间接关联。
|
||||
- 字段要点：
|
||||
- 渠道批次与模板：`channel_activity_id`（渠道立减金批次号）、`temp_no`（券模板编号，仅支付宝）。
|
||||
- 服务商：`provider`（微信/支付宝可选）。
|
||||
- 权重：`weight`（用于批次选择或优先级控制）。
|
||||
- 时间：`create_time`、`update_time`。
|
||||
- 索引：
|
||||
- 现有：唯一 `channel_activity_id`。
|
||||
- 建议：如按 `voucher_id` 维度频繁查询，增加 `voucher_id` 索引；列表按时间查询可增加 `create_time` 索引。
|
||||
- 导出查询规范：
|
||||
- 批次信息通常作为维表使用，导出应通过与 `voucher` 或 `order_voucher` 关联来继承权限边界（如订单的 `creator IN (...)`）。
|
||||
- 字段白名单：默认允许 `voucher_id/channel_activity_id/provider/temp_no/weight/create_time/update_time`；避免导出仅内部使用的渠道敏感标识到公共模板。
|
||||
- EXPLAIN 达标建议：
|
||||
- 关联查询通过唯一或索引列（`channel_activity_id`/`voucher_id`）点查；列表查询按时间索引并分页，避免 `Using filesort/temporary`。
|
||||
- 安全与合规：
|
||||
- 渠道模板编号与服务商标识可能为内部标识；默认仅在具备相应权限的模板中使用，导出需审计。
|
||||
|
||||
## 28. 营销系统表关系与映射
|
||||
|
||||
- 驱动表与权限：
|
||||
- 驱动表：`order`（主键 `order_number`，权限字段 `creator`）。
|
||||
- 所有导出查询以 `order` 为驱动，强制注入权限过滤：`creator IN (...)`，并结合时间范围 `create_time`。
|
||||
- 关系与连接（均采用 `LEFT JOIN`，仅当业务语义要求强关联时可用 `INNER JOIN`）：
|
||||
- 详情：`order_detail.order_number = order.order_number`。
|
||||
- 红包：`order_cash.order_number = order.order_number`。
|
||||
- 立减金订单：`order_voucher.order_number = order.order_number`。
|
||||
- 立减金：`voucher.channel_activity_id = order_voucher.channel_activity_id`。
|
||||
- 立减金批次：`voucher_batch.voucher_id = voucher.id`。
|
||||
- 计划：`plan.id = order.plan_id`。
|
||||
- KEY 批次：`key_batch.plan_id = plan.id`。
|
||||
- 兑换码批次：`code_batch.key_batch_id = key_batch.id`。
|
||||
- 开放平台发放记录：`` `order`.`key` = merchant_key_send.key ``。
|
||||
- 连接键索引与保留字：
|
||||
- 为上述连接键建立或验证必要索引：`order.order_number`、`order.creator, order.create_time`、`order_detail.order_number`、`order_cash.order_number`、`order_voucher.order_number`、`voucher.channel_activity_id`、`voucher_batch.voucher_id`、`plan.id`、`key_batch.plan_id`、`code_batch.key_batch_id`、`merchant_key_send.key`。
|
||||
- 表名 `order` 与列名 `` `key` ``为保留字，SQL 中需使用反引号包裹以避免语法冲突。
|
||||
- 模板生成 SQL 的连接顺序建议：
|
||||
- 优先应用驱动表过滤（`creator + create_time + type/...`），再按选择性从高到低依次连接：计划 → KEY 批次 → 兑换码批次；立减金订单 → 立减金 → 立减金批次；红包；详情；发放记录。
|
||||
- 避免无条件连接造成笛卡尔积；每个连接必须具备明确的等值连接键。
|
||||
- 性能与 EXPLAIN 要点：
|
||||
- 连接计划应使高选择性条件尽早生效；排序字段尽量与索引前缀一致。
|
||||
- 对超大数据场景采用分页与稳定排序键；避免在大范围上出现 `Using temporary` 与 `Using filesort`。
|
||||
- 字段白名单与敏感字段：
|
||||
- 默认排除敏感列（如 `card_code`、账号与渠道标识、JSON 大字段）；如需导出需更高权限并进行掩码或选择性键导出。
|
||||
- 前端模板仅提供白名单字段选择，禁止自由输入列名与表名。
|
||||
|
||||
## 29. 操作面板与模板生成流程
|
||||
|
||||
- 面板步骤：
|
||||
- 选择数据源：`营销系统` 或 `易码通`（当前默认支持“订单数据”场景）。
|
||||
- 选择导出场景：`订单数据`（后续可扩展其他场景）。
|
||||
- 选择附表信息：依据“表关系与映射”显示可选附表与连接键；用户勾选所需的详情、红包、立减金、计划、KEY 批次、兑换码批次、发放记录等。
|
||||
- 设置权限范围与条件：必填 `creator IN (...)`（或租户范围）与时间窗口（如 `order.create_time`）；可选类型、状态、渠道、计划等条件；所有条件走参数化。
|
||||
- 选择字段与输出格式：仅白名单字段；输出格式 `csv/xlsx`；是否列统计、是否按条件分多 sheet。
|
||||
- 确认生成模板：生成模板并运行 `EXPLAIN` 评估，达标后方可保存与执行。
|
||||
- SQL 生成：
|
||||
- 以驱动表 `order` 构建 `SELECT`；按已选附表生成 `LEFT JOIN`，连接条件来自“表关系与映射”。
|
||||
- 强制注入权限与时间条件；仅允许白名单列与等值/范围过滤；禁止自由拼接 SQL 与 `SELECT *`。
|
||||
- 排序与分页：按索引前缀字段排序；大数据导出采用分页与稳定排序键；尽量一次查询覆盖需求并流式写文件。
|
||||
- 多 sheet：可按指定列（如订单类型或日期）拆分；保持稳定排序避免跨批次混入。
|
||||
- 列统计：在同一 SQL 使用聚合/窗口生成统计或追加统计 sheet，避免二次扫描。
|
||||
- EXPLAIN 分析面板：
|
||||
- 展示核心字段：`id`、`select_type`、`table`、`type`、`possible_keys`、`key`、`key_len`、`ref`、`rows`、`filtered`、`Extra`。
|
||||
- 评估规则：禁止大表 `type=ALL`；优先 `ref/range/const`；`rows` 合理可控；避免超大数据集上的 `Using temporary/filesort`。
|
||||
- 达标门槛：满足上述规则后允许执行；超标给出索引建议与模板调整指引（减少字段、改时间窗口、补充过滤或新增索引建议）。
|
||||
- 模板保存与可见性：
|
||||
- 保存为 `export_templates` 记录，包含数据源、主表、关联、字段、过滤、格式、统计、多 sheet、可见性（公共/个人）、所有者、`EXPLAIN` 结果与评分。
|
||||
- 公共模板需通过评审并标注责任人；个人模板仅本人可见或按角色授权可见。
|
||||
- 执行与进度：
|
||||
- 发起导出后记录到 `export_jobs`；展示总行数估算/实际、当前批次、速率、预计完成时间与状态；支持取消与失败重试。
|
||||
- 文件记录存入 `export_job_files`，下载链接带签名与有效期；支持 `csv/xlsx`。
|
||||
- 安全与合规：
|
||||
- 敏感字段默认不可选（如 `card_code`、账号、渠道标识与大 JSON）；需要更高权限并进行掩码或选择性键导出。
|
||||
- 所有参数化与权限注入由后端统一实现；导出与下载行为记录审计。
|
||||
|
||||
## 30. 字段选择与列名约定
|
||||
|
||||
- 默认选择：
|
||||
- 默认仅导出主表 `order` 的白名单字段；附表字段需用户显式勾选。
|
||||
- 默认排除敏感字段（如 `card_code`、账号标识、渠道标识、大 JSON 等）。
|
||||
- 列名来源：
|
||||
- 优先使用数据库字段的 `COMMENT` 作为导出列名（中文优先）。
|
||||
- 如 `COMMENT` 为空或过于技术化，则：
|
||||
- 使用模板中的别名作为列名；
|
||||
- 或将列名从 `snake_case`/`camelCase` 转为可读名（如 `order_number` → `订单编号`，无词典时保留原名）。
|
||||
- 冲突与去重：
|
||||
- 当不同表存在同名字段并被选择时，自动加前缀（如 `订单.订单编号`、`计划.计划编号`）或根据模板别名消除歧义。
|
||||
- 同一语义字段重复选择时进行去重或保留一个主列，遵循模板优先级。
|
||||
- 排序与输出：
|
||||
- 列顺序遵循用户选择顺序；主表字段在前，附表字段在后；统计列或多 sheet 信息追加在末尾或独立 sheet。
|
||||
- 文本统一 `UTF-8`；数值与时间按统一格式输出（时间默认 `YYYY-MM-DD HH:mm:ss`，时区统一为 `UTC` 或模板指定）。
|
||||
- 安全与合规：
|
||||
- 对被允许导出的敏感列进行掩码（如账号、姓名仅展示部分）；JSON 字段仅允许按键路径选择性导出。
|
||||
- 列名中不包含敏感信息或内部标识；对渠道或凭据相关内容默认不展示于列名与文件中。
|
||||
|
||||
## 31. 新增导出模板整体流程
|
||||
|
||||
- 新增导出：在操作面板点击“新增导出”。
|
||||
- 选择数据源：`营销系统` 或 `易码通`。
|
||||
- 选择导出场景：当前支持 `订单数据`（后续可扩展）。
|
||||
- 自定义导出字段：根据“表关系与映射”逐步选择主表与附表字段；默认仅主表字段，附表需显式勾选；仅白名单字段。
|
||||
- 生成模板：点击“生成模板”，后端构建 SQL 并执行 `EXPLAIN` 分析。
|
||||
- 返回分析结果：展示 `EXPLAIN` 关键项与评估结论；若达标则保存模板并可执行导出；未达标返回索引建议与模板调整提示。
|
||||
- 执行导出：可立即基于模板发起导出，进入进度追踪与文件下载流程。
|
||||
|
||||
## 32. 模板列表与检索
|
||||
|
||||
- 列表范围：展示“内置公共模板”与“当前账号所有模板”（个人/共享）。管理员或具备相应角色可查看全部模板。
|
||||
- 基本展示列：
|
||||
- 模板名称、数据源（营销系统/易码通）、场景（订单数据）、主表、可见性（公共/个人）、所有者、创建/更新时间。
|
||||
- EXPLAIN 评分与最近校验时间、字段数量、关联表数量、输出格式（csv/xlsx）、统计开启与多 sheet 条件。
|
||||
- 过滤条件：
|
||||
- 数据源、场景、可见性、所有者、创建时间范围、启用状态、EXPLAIN 评分区间、标签（如渠道/计划/类型）。
|
||||
- 关键字搜索支持名称与主表/关联表名，采用前缀匹配与分页限制。
|
||||
- 排序与分页：
|
||||
- 支持按更新时间、创建时间、EXPLAIN 评分、预计行数等排序；统一服务端分页与总数统计。
|
||||
- 快捷操作：
|
||||
- 预览 SQL（敏感信息掩码）、重新校验 EXPLAIN、编辑元数据、启用/停用、发起导出、查看历史下载、复制（fork）模板、删除（仅未被引用或无权限边界影响时）。
|
||||
- 权限与可见性：
|
||||
- 公共模板所有符合角色/租户的用户可见与使用；个人模板仅所有者或授权角色可见；维护者可在可见范围内编辑公共模板。
|
||||
- 删除受限：被导出记录引用的模板不可直接删除（或需先停用并解除引用）。
|
||||
- 安全与审计：
|
||||
- 列表与详情不展示完整 DSN 或任何敏感凭据；SQL 预览中掩码权限范围与敏感列。
|
||||
- 模板的启停、删除、复制、校验与发起导出操作均记录审计日志。
|
||||
|
||||
## 33. 导出记录查看与操作
|
||||
|
||||
- 展示内容：
|
||||
- 文件大小（`size_bytes` 聚合显示与各文件明细）。
|
||||
- 文件地址（`storage_uri`，展示可下载的签名链接）。
|
||||
- 数据量大小（总行数与各文件 `row_count`）。
|
||||
- 导出开始时间（`started_at`）与完成时间（`finished_at`）。
|
||||
- 执行人 ID（`requested_by`）。
|
||||
- 任务状态（`queued/running/failed/completed/canceled`）与耗时。
|
||||
- 列表来源：
|
||||
- 来自 `export_jobs` 与 `export_job_files` 的聚合视图；多文件（多 sheet 或分块）展示合计与明细展开。
|
||||
- 过滤与排序：
|
||||
- 按执行人、时间范围、状态、模板、数据源过滤；按完成时间、耗时、文件大小、行数排序。
|
||||
- 服务端分页与总数统计；支持关键字搜索模板名称与执行人。
|
||||
- 行操作：
|
||||
- 下载：提供带签名与有效期的下载链接；支持单文件与批量（ZIP 封装）。
|
||||
- 删除：
|
||||
- 仅拥有者或具备相应角色可删除；公共任务需具备维护权限。
|
||||
- 默认删除逻辑：删除文件的物理存储并保留 `export_jobs` 元数据与审计；或将记录标记为已清理。
|
||||
- 对正在运行的任务不可删除；对失败任务可清理文件记录与中间产物。
|
||||
- 权限与合规：
|
||||
- 下载链接签名与有效期控制，防止外泄；公开范围需受角色与租户限制。
|
||||
- 列表中不展示敏感参数；文件名与路径不包含凭据或内部标识。
|
||||
- 审计与保留：
|
||||
- 下载与删除操作记录审计（操作者、时间、目标文件、结果）。
|
||||
- 数据保留策略：按时间与空间配额定期清理历史文件，仅保留元数据；可配置每模板或每执行人的保留上限。
|
||||
|
||||
## 34. 模板行执行导出与进度展示
|
||||
|
||||
- 触发入口：
|
||||
- 模板列表的每一行提供“执行导出”按钮；点击后创建一条新的导出任务记录并异步执行。
|
||||
- 任务创建：
|
||||
- 在 `export_jobs` 新增记录：`template_id`、`requested_by`、`permission_scope_json`、`options_json`、`file_format`、`status=queued`、`created_at`。
|
||||
- 初始化 `row_estimate`（来自 `EXPLAIN` 或估算流程）、`started_at`（排队开始）、`total_rows`（执行中累计）。
|
||||
- 异步执行：
|
||||
- 任务状态流转：`queued → running → (completed | failed | canceled)`；支持取消与失败后重试。
|
||||
- 并发与限流：对全局与每账号的并发进行限流与队列调度，避免资源争抢。
|
||||
- 预计完成时间（ETA）：
|
||||
- 依据 `row_estimate` 与实时写入速率（行/秒）计算 ETA，并随进度动态更新；若估算缺失，先以启动阶段速率预估后修正。
|
||||
- 进度展示：
|
||||
- 实时显示：已写行数、当前批次、平均速率、ETA、状态与错误摘要。
|
||||
- 推送方式：SSE/WebSocket 或按 1-3 秒轮询；界面在模板行与导出记录页面均显示进度。
|
||||
- 完成反馈：
|
||||
- 导出完成后弹出通知，提供下载入口；同时在“导出记录”页可下载。
|
||||
- 文件记录：在 `export_job_files` 新增记录并展示 `storage_uri`（签名链接）、`row_count`、`size_bytes`、`sheet_name`（如多 sheet）。
|
||||
- 安全与合规：
|
||||
- 下载链接签名与有效期；权限校验基于模板的可见范围与执行人身份。
|
||||
- 进度与日志不展示敏感参数与凭据；错误信息进行安全脱敏。
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
FROM golang:1.21-alpine AS build
|
||||
WORKDIR /app
|
||||
RUN --mount=type=cache,target=/var/cache/apk apk add build-base
|
||||
COPY server/ ./server/
|
||||
COPY web/ ./web/
|
||||
WORKDIR /app/server
|
||||
RUN go build -o /app/bin/marketing-data-server ./cmd/server
|
||||
|
||||
FROM alpine:3.19
|
||||
WORKDIR /app
|
||||
COPY --from=build /app/bin/marketing-data-server /app/bin/marketing-data-server
|
||||
COPY --from=build /app/web /app/web
|
||||
RUN mkdir -p /app/storage/export
|
||||
EXPOSE 8077
|
||||
ENTRYPOINT ["/app/bin/marketing-data-server"]
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
# MarketingSystemDataTool
|
||||
|
||||
营销系统、易码通数据工具
|
||||
|
||||
## 服务启动方式
|
||||
|
||||
### 后端服务
|
||||
|
||||
1. **编译服务**（如未编译）：
|
||||
```bash
|
||||
cd server
|
||||
go build -o server ./cmd/server/main.go
|
||||
```
|
||||
|
||||
2. **启动服务**：
|
||||
```bash
|
||||
cd server
|
||||
./server
|
||||
```
|
||||
|
||||
3. **服务配置**：
|
||||
- 配置文件：`server/config.yaml`
|
||||
- 默认端口：8077
|
||||
- 可通过环境变量覆盖配置
|
||||
|
||||
### 前端访问
|
||||
|
||||
服务启动后，通过浏览器访问：
|
||||
```
|
||||
http://localhost:8077
|
||||
```
|
||||
|
||||
## 技术栈
|
||||
|
||||
- 后端：Go 1.21
|
||||
- 前端：Vue 3 + Element Plus（通过 CDN 引入）
|
||||
- 数据库：MySQL
|
||||
- 导出格式：CSV、Excel
|
||||
|
||||
## 项目结构
|
||||
|
||||
- `server/`：Go 服务端代码
|
||||
- `web/`：前端页面与静态资源
|
||||
- `config/`：非敏感配置
|
||||
- `scripts/`：开发与运维脚本
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
{
|
||||
"tables": ["order"],
|
||||
"fields": [
|
||||
"order.order_number","order.creator","order.out_trade_no","order.type","order.status","order.contract_price","order.num","order.total","order.pay_amount","order.create_time","order.update_time",
|
||||
"order_detail.plan_title","order_detail.reseller_name","order_detail.product_name","order_detail.show_url","order_detail.official_price","order_detail.cost_price","order_detail.create_time","order_detail.update_time",
|
||||
"order_cash.channel","order_cash.cash_activity_id","order_cash.receive_status","order_cash.receive_time","order_cash.cash_packet_id","order_cash.cash_id","order_cash.amount","order_cash.status","order_cash.expire_time","order_cash.update_time",
|
||||
"order_voucher.channel","order_voucher.channel_activity_id","order_voucher.channel_voucher_id","order_voucher.status","order_voucher.grant_time","order_voucher.usage_time","order_voucher.refund_time","order_voucher.status_modify_time","order_voucher.overdue_time","order_voucher.refund_amount","order_voucher.official_price","order_voucher.out_biz_no","order_voucher.account_no",
|
||||
"plan.id","plan.title","plan.status","plan.begin_time","plan.end_time",
|
||||
"key_batch.id","key_batch.batch_name","key_batch.bind_object","key_batch.quantity","key_batch.stock","key_batch.begin_time","key_batch.end_time",
|
||||
"code_batch.id","code_batch.title","code_batch.status","code_batch.begin_time","code_batch.end_time","code_batch.quantity","code_batch.usage","code_batch.stock",
|
||||
"voucher.channel","voucher.channel_activity_id","voucher.price","voucher.balance","voucher.used_amount","voucher.denomination",
|
||||
"voucher_batch.channel_activity_id","voucher_batch.temp_no","voucher_batch.provider","voucher_batch.weight",
|
||||
"merchant_key_send.merchant_id","merchant_key_send.out_biz_no","merchant_key_send.key","merchant_key_send.status","merchant_key_send.usage_time","merchant_key_send.create_time"
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,144 @@
|
|||
# MarketingSystemDataTool 需求文档与测试点
|
||||
|
||||
## 背景与目标
|
||||
- 为营销系统与易码通系统提供统一、可配置的数据导出能力（CSV/XLSX），以模板化方式生成安全、可控的查询与导出文件。
|
||||
- 支持模板管理、EXPLAIN 评估、分批与时间分片执行、进度追踪与文件下载，满足运营与数据分析的批量导出场景。
|
||||
|
||||
## 范围
|
||||
- 后端：Go 单服务，HTTP API，连接两套 MySQL（营销、易码通），不含消息队列与缓存。
|
||||
- 前端：Vue 3 + Element Plus（CDN），单页工具页面，模板维护与导出发起、任务查看与下载。
|
||||
- 存储：本地 `storage/export/` 目录生成分片文件与最终 zip；导出记录与模板元数据存库。
|
||||
|
||||
## 系统架构
|
||||
- 路由注册：`server/internal/api/router.go:11`、`server/internal/api/router.go:13`、`server/internal/api/router.go:15`、`server/internal/api/router.go:16-27`；静态文件：`server/internal/api/router.go:40`。
|
||||
- 启动入口：`server/cmd/server/main.go:15-46`，加载配置、初始化日志、连接 MySQL、启动 HTTP Server。
|
||||
- 配置加载：`server/internal/config/config.go:31-69`（YAML + 环境变量），DSN 组合：`server/internal/config/config.go:94-99`。
|
||||
- 响应封装与中间件：`server/internal/api/response.go:18-36`、`server/internal/api/middleware.go:18-31`（TraceID/CORS/访问日志）。
|
||||
- SQL 构建与评估：`server/internal/exporter/sqlbuilder.go`、`server/internal/exporter/explain.go`、`server/internal/exporter/evaluate.go`、`server/internal/exporter/writer.go`。
|
||||
- Schema 映射与白名单：`server/internal/schema/*.go`；字段标签：`server/internal/api/templates.go:315-317`。
|
||||
- 前端页面与逻辑：`web/index.html`、`web/main.js`。
|
||||
|
||||
## 功能需求
|
||||
### 模板管理
|
||||
- 新增模板：`POST /api/templates`，字段包含名称、数据源、场景主表、字段集合、默认筛选、输出格式、可见性、归属人。
|
||||
- 列表与详情：`GET /api/templates`、`GET /api/templates/{id}`，支持 `userId` 过滤公共/个人模板；展示字段数、执行次数、最近校验。
|
||||
- 编辑模板：`PATCH /api/templates/{id}`，支持名称、数据源、主表、字段、筛选、格式、可见性、启用状态变更。
|
||||
- 删除模板：`DELETE /api/templates/{id}`，若存在任务引用禁止删除。
|
||||
- 模板校验：`POST /api/templates/{id}/validate`，生成 SQL、执行 EXPLAIN、评分与索引建议写回模板。
|
||||
|
||||
### 导出执行
|
||||
- 发起导出：`POST /api/exports`，加载模板→构建 SQL→EXPLAIN 评分（阈值 60）→估算行数→入库任务→异步执行。
|
||||
- 进度与列表：`GET /api/exports` 分页返回任务，含评估摘要与行数；`GET /api/exports/{id}` 返回详细信息与文件列表。
|
||||
- SQL 查看：`GET /api/exports/{id}/sql` 返回参数化 SQL 与最终展开 SQL。
|
||||
- 取消与下载：`POST /api/exports/{id}/cancel`、`GET /api/exports/{id}/download` 下载最新 zip。
|
||||
|
||||
### 元数据与辅助
|
||||
- 字段元数据：`GET /api/metadata/fields?datasource&order_type`，返回表与字段、推荐字段集合。
|
||||
- 选择项：营销侧 `GET /api/creators`、`GET /api/resellers?creator`、`GET /api/plans?reseller`；易码通 `GET /api/ymt/users`、`GET /api/ymt/merchants?user_id`、`GET /api/ymt/activities?merchant_id`。
|
||||
- Key 解析工具：`GET /api/utils/decode_key?v=...`。
|
||||
|
||||
### 权限与安全
|
||||
- 强制权限条件：营销库必须提供 `creator_in` 与 `create_time_between`；易码通通过 `creator_in`（映射为 `user_id`）。
|
||||
- 字段白名单：禁用 `SELECT *` 与非白名单字段；模板字段、过滤项统一白名单校验。
|
||||
- 参数化查询：所有筛选采用预编译参数；禁止字符串拼接。
|
||||
- 敏感字段处理：`order.key` 在易码通解密（`SM4`，`YMT_KEY_DECRYPT_KEY_B64`），营销系统做逆编码；下载统一 `zip`。
|
||||
|
||||
### 文件生成与存储
|
||||
- 输出格式：`csv` 与 `xlsx`；首行写列名；统一 UTF-8。
|
||||
- 分批与分片：每批查询 `1000` 行；单文件上限 `300000` 行自动切片；支持按时间范围（10 天步长）拆分执行。
|
||||
- 记录文件：每个分片与最终 zip 写入 `export_job_files`，保留行数与大小。
|
||||
|
||||
### 前端交互
|
||||
- 模板管理与执行：`web/index.html`、`web/main.js`；通过 `userId` 透传归属过滤；Cascader 选择场景字段；对话框尺寸可调整。
|
||||
- 执行表单：必填时间范围；营销侧创建者/分销商/计划级联；易码通用户/客户/活动级联；订单类型附加筛选。
|
||||
- 任务列表：每秒轮询；展示评估状态、行数、进度百分比、下载与 SQL 分析。
|
||||
|
||||
## 非功能需求
|
||||
- 性能：EXPLAIN 评分达标（≥60）；避免 `ALL`、`Using temporary/filesort`；优先覆盖权限与时间索引。
|
||||
- 稳定性：分批拉取与时间分片；进度每 50 行刷新；失败/取消状态管理；磁盘 I/O 控制。
|
||||
- 安全：仅白名单字段；敏感信息通过环境变量注入；日志不打印明文密码与完整 DSN。
|
||||
- 监控与日志：统一结构化访问日志、错误日志携带 TraceID 与 SQL；任务与模板操作记录。
|
||||
|
||||
## 接口规范（摘要）
|
||||
- `POST /api/templates`
|
||||
- 请求：`{ name, datasource, main_table, fields: string[], filters: object, file_format, visibility, owner_id }`
|
||||
- 响应：`201 ok`
|
||||
- `GET /api/templates[?userId]` → 列表；`GET /api/templates/{id}` → 详情
|
||||
- `PATCH /api/templates/{id}` → 部分更新
|
||||
- `DELETE /api/templates/{id}` → 模板未被引用时允许删除
|
||||
- `POST /api/templates/{id}/validate` → `{ score, suggestions[] }`
|
||||
- `POST /api/exports`
|
||||
- 请求：`{ template_id, requested_by, permission, options, file_format, filters, datasource }`
|
||||
- 响应：`{ id }`（任务 ID）
|
||||
- `GET /api/exports[?template_id&page&page_size&userId]` → `{ items, total, page, page_size }`
|
||||
- `GET /api/exports/{id}` → 任务详情与文件列表
|
||||
- `GET /api/exports/{id}/sql` → `{ sql, final_sql }`
|
||||
- `POST /api/exports/{id}/cancel`、`GET /api/exports/{id}/download`
|
||||
- `GET /api/metadata/fields?datasource&order_type` → `{ tables[], recommended[] }`
|
||||
|
||||
## 配置与环境
|
||||
- YAML：`server/config.yaml`（或根 `config.yaml`），示例端口 `8077`；两套 DB 连接参数与易码通密钥。
|
||||
- 环境变量覆盖：`MARKETING_DB_*`、`YMT_DB_*`、`YMT_KEY_DECRYPT_KEY_B64`；支持 `.env.local` 注入。
|
||||
- DSN 模板：`<user>:<password>@tcp(<host>:<port>)/<dbname>?parseTime=True&loc=Local&charset=utf8mb4`。
|
||||
|
||||
## 数据与字段（摘要）
|
||||
- 主表：营销 `order`；易码通主表映射 `order_info`。
|
||||
- 常用过滤：`creator_in`、`create_time_between`、`type_eq`、`out_trade_no_eq`、`plan_id_eq`、`reseller_id_eq` 等。
|
||||
- 关联表：营销（`order_detail`、`order_cash`、`order_voucher`、`plan`、`key_batch`、`code_batch`、`voucher`、`voucher_batch`、`merchant_key_send`）；易码通（`order_cash`、`order_voucher`、`order_digit`、`goods_voucher_batch`、`goods_voucher_subject_config`、`merchant`、`activity`）。
|
||||
|
||||
## 关键流程（文字版）
|
||||
1. 前端创建模板或选择已有模板，选择数据源、场景与字段，设置默认订单类型。
|
||||
2. 发起导出：前端提交必填的时间范围与权限范围（创建者等），后端生成 SQL → EXPLAIN → 阈值校验。
|
||||
3. 任务入库与异步执行：分批分页查询，每 1000 行拉取，单文件 `300k` 行分片，时间范围大时按 10 天拆分。
|
||||
4. 行写入：写 CSV/XLSX；特定字段转换（`order.key` 解密/逆编码）；定期更新进度。
|
||||
5. 完成：汇总分片为 zip，写文件记录，任务置为 `completed`，提供下载与 SQL 分析。
|
||||
|
||||
## 边界与限制
|
||||
- 仅支持订单主表（营销 `order`，易码通 `order_info`），不支持自由主表选择。
|
||||
- 必须包含权限与时间过滤；否则构建器报错或评估不通过。
|
||||
- 单文件分片上限 `300k` 行；极大数据量建议按时间拆分或筛选范围收窄。
|
||||
|
||||
## 测试点
|
||||
### 单元测试
|
||||
- SQL 构建器（`server/internal/exporter/sqlbuilder.go`）
|
||||
- 字段白名单与非法字段拒绝。
|
||||
- 主表校验：仅 `order` / `order_info`。
|
||||
- 过滤解析：`creator_in`（多类型数组）、`create_time_between`（长度=2）、其他等值过滤。
|
||||
- 易码通字段映射与特殊枚举列（`type/status/pay_status` CASE 逻辑）。
|
||||
- JOIN 选择与避免笛卡尔积（不同过滤与字段组合）。
|
||||
- EXPLAIN 评估（`server/internal/exporter/explain.go`、`evaluate.go`）
|
||||
- 出现 `ALL`/高 `rows`/`Using temporary/filesort` 的扣分逻辑与建议生成。
|
||||
- 评分阈值判定与错误分支（禁止执行）。
|
||||
- 写入器（`server/internal/exporter/writer.go`）
|
||||
- CSV/XLSX 写列名与行；`Close` 返回路径与大小；异常路径处理。
|
||||
- 行转换（`server/internal/api/exports.go:868-887`）
|
||||
- `order.key` 解密/逆编码：易码通 SM4（密钥为空与有效）、营销逆编码后长度与字符集校验。
|
||||
- 时间分片（`server/internal/api/exports.go:731-751`）
|
||||
- 输入合法性与步长切分边界；起止相等、跨天与不足步长。
|
||||
|
||||
### 集成测试
|
||||
- 模板生命周期：创建→校验→编辑→列表/详情→删除（被引用场景拒绝）。
|
||||
- 导出端到端：发起（营销与易码通各 1 套）→下载→SQL 分析；大窗口下分片与 zip 生成验证。
|
||||
- 列表分页与过滤：`/api/exports` `page/page_size/template_id/userId` 组合下返回一致性。
|
||||
- 前端交互：字段级联选择、对话框动态尺寸、`userId` 透传与权限限制（编辑/删除按钮显隐）。
|
||||
|
||||
### 性能测试
|
||||
- EXPLAIN 评分边界：构造不同索引覆盖与过滤条件，验证评分变化与建议内容。
|
||||
- 大数据量导出：行数 10 万/30 万/100 万分批写入，评估速度、文件大小与 zip 打包时间；进度刷新频率与准确性。
|
||||
|
||||
### 安全测试
|
||||
- 非白名单字段与非法主表拒绝；SQL 注入尝试（字符串拼接禁止）。
|
||||
- 敏感信息：`order.key` 转换正确且日志不泄漏；配置与环境变量加载不写入日志明文密码。
|
||||
- 下载鉴权：仅通过最新文件下载 API，404 场景与异常路径。
|
||||
|
||||
### 前端测试
|
||||
- 表单校验：模板创建必填项、导出必填时间范围与权限范围。
|
||||
- 选择项联动：创建者→分销商→计划（营销）；用户→客户→活动（易码通）。
|
||||
- 任务轮询：进度计算、完成后下载按钮显隐；SQL 查看正确展示最终 SQL。
|
||||
|
||||
## 验收标准
|
||||
- 模板创建、校验、编辑、删除流程完整可用；字段白名单生效。
|
||||
- 导出任务在营销与易码通数据源下均可成功执行，EXPLAIN 评分≥60，能生成 zip 文件并下载。
|
||||
- 在大窗口与分片场景下保持稳定，进度与行数统计准确；日志与响应携带 TraceID。
|
||||
- 前端交互顺畅，必填项校验与权限控制符合预期；SQL 分析与下载功能可用。
|
||||
|
||||
|
|
@ -0,0 +1,38 @@
|
|||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ROOT_DIR="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
ENV_NAME="${1:-test}"
|
||||
IMAGE="marketing-system-data-tool"
|
||||
TAG="$ENV_NAME"
|
||||
PORT="${PORT:-8077}"
|
||||
cd "$ROOT_DIR"
|
||||
if docker image inspect "$IMAGE:$TAG" >/dev/null 2>&1; then
|
||||
echo "使用已有镜像作为缓存: $IMAGE:$TAG"
|
||||
DOCKER_BUILDKIT=1 docker build --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from "$IMAGE:$TAG" -t "$IMAGE:$TAG" -f Dockerfile .
|
||||
else
|
||||
echo "镜像不存在，开始构建: $IMAGE:$TAG"
|
||||
DOCKER_BUILDKIT=1 docker build --build-arg BUILDKIT_INLINE_CACHE=1 -t "$IMAGE:$TAG" -f Dockerfile .
|
||||
fi
|
||||
mkdir -p log storage/export
|
||||
CID_NAME="marketing-data-$ENV_NAME"
|
||||
RUNNING=$(docker inspect -f '{{.State.Running}}' "$CID_NAME" 2>/dev/null || echo false)
|
||||
if [ "$RUNNING" = "true" ]; then
|
||||
docker stop "$CID_NAME" >/dev/null 2>&1 || true
|
||||
fi
|
||||
if docker ps -a --format '{{.Names}}' | grep -q "^${CID_NAME}$"; then
|
||||
docker rm "$CID_NAME" >/dev/null 2>&1 || true
|
||||
fi
|
||||
CONFIG_PATH="$ROOT_DIR/server/config.yaml"
|
||||
if [ ! -f "$CONFIG_PATH" ]; then
|
||||
echo "配置文件缺失：$CONFIG_PATH" >&2
|
||||
exit 1
|
||||
fi
|
||||
docker run -d \
|
||||
--name "$CID_NAME" \
|
||||
--restart unless-stopped \
|
||||
-p "$PORT:8077" \
|
||||
-v "$ROOT_DIR/storage/export:/app/storage/export" \
|
||||
-v "$ROOT_DIR/log:/app/log" \
|
||||
-v "$CONFIG_PATH:/app/server/config.yaml:ro" \
|
||||
"$IMAGE:$TAG"
|
||||
echo "container: $CID_NAME image: $IMAGE:$TAG port: $PORT"
|
||||
Binary file not shown.
|
|
@ -0,0 +1,63 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"server/internal/api"
|
||||
"server/internal/config"
|
||||
"server/internal/db"
|
||||
"server/internal/logging"
|
||||
"time"
|
||||
)
|
||||
|
||||
func main() {
|
||||
cfg := config.Load()
|
||||
if cfg.YMTKeyDecryptKeyB64 != "" {
|
||||
os.Setenv("YMT_KEY_DECRYPT_KEY_B64", cfg.YMTKeyDecryptKeyB64)
|
||||
}
|
||||
_ = logging.Init("log")
|
||||
log.Println("connecting YMT MySQL:", cfg.YMTDB.Host+":"+cfg.YMTDB.Port, "db", cfg.YMTDB.Name, "user", cfg.YMTDB.User)
|
||||
log.Println("connecting Marketing MySQL:", cfg.MarketingDB.Host+":"+cfg.MarketingDB.Port, "db", cfg.MarketingDB.Name, "user", cfg.MarketingDB.User)
|
||||
log.Println("connecting Meta MySQL (templates/jobs):", cfg.YMTTestDB.Host+":"+cfg.YMTTestDB.Port, "db", cfg.YMTTestDB.Name, "user", cfg.YMTTestDB.User)
|
||||
if cfg.YMTDB.DSN() == "" {
|
||||
log.Fatal("YMT MySQL DSN missing: host, port, user, name required")
|
||||
}
|
||||
if cfg.MarketingDB.DSN() == "" {
|
||||
log.Fatal("Marketing MySQL DSN missing: host, port, user, name required")
|
||||
}
|
||||
if cfg.YMTTestDB.DSN() == "" {
|
||||
log.Fatal("Meta MySQL DSN missing: host, port, user, name required")
|
||||
}
|
||||
ymt, err := db.ConnectMySQL(cfg.YMTDB.DSN())
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
// apply pool settings from env for YMT
|
||||
db.ApplyPoolFromEnv(ymt, "YMT_DB_")
|
||||
marketing, err := db.ConnectMySQL(cfg.MarketingDB.DSN())
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
// apply pool settings from env for Marketing
|
||||
db.ApplyPoolFromEnv(marketing, "MARKETING_DB_")
|
||||
meta, err := db.ConnectMySQL(cfg.YMTTestDB.DSN())
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
// apply pool settings from env for Meta (templates/jobs)
|
||||
db.ApplyPoolFromEnv(meta, "YMT_TEST_DB_")
|
||||
r := api.NewRouter(meta, marketing, ymt)
|
||||
addr := ":" + func() string {
|
||||
s := cfg.Port
|
||||
if s == "" {
|
||||
return "8077"
|
||||
}
|
||||
return s
|
||||
}()
|
||||
srv := &http.Server{Addr: addr, Handler: r, ReadTimeout: 15 * time.Second, WriteTimeout: 60 * time.Second}
|
||||
log.Println("server listening on ", addr)
|
||||
log.Fatal(srv.ListenAndServe())
|
||||
}
|
||||
|
||||
// buildDSN deprecated; use cfg.YMTDB.DSN()/cfg.MarketingDB.DSN()
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
app:
|
||||
port: "8077"
|
||||
marketing_db:
|
||||
host: "YOUR_MARKETING_DB_HOST"
|
||||
port: "3306"
|
||||
user: "YOUR_MARKETING_DB_USER"
|
||||
password: "YOUR_MARKETING_DB_PASSWORD"
|
||||
name: "YOUR_MARKETING_DB_NAME"
|
||||
ymt_db:
|
||||
host: "YOUR_YMT_DB_HOST"
|
||||
port: "3306"
|
||||
user: "YOUR_YMT_DB_USER"
|
||||
password: "YOUR_YMT_DB_PASSWORD"
|
||||
name: "YOUR_YMT_DB_NAME"
|
||||
ymt_key_decrypt_key_b64: ""
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
module server
|
||||
|
||||
go 1.21
|
||||
|
||||
require (
|
||||
github.com/go-sql-driver/mysql v1.7.1
|
||||
github.com/xuri/excelize/v2 v2.8.1
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 // indirect
|
||||
github.com/richardlehane/mscfb v1.0.4 // indirect
|
||||
github.com/richardlehane/msoleps v1.0.3 // indirect
|
||||
github.com/tjfoc/gmsm v1.4.1 // indirect
|
||||
github.com/xuri/efp v0.0.0-20231025114914-d1ff6096ae53 // indirect
|
||||
github.com/xuri/nfp v0.0.0-20230919160717-d98342af3f05 // indirect
|
||||
golang.org/x/crypto v0.19.0 // indirect
|
||||
golang.org/x/net v0.21.0 // indirect
|
||||
golang.org/x/text v0.14.0 // indirect
|
||||
)
|
||||
|
|
@ -0,0 +1,104 @@
|
|||
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
|
||||
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
|
||||
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
|
||||
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
|
||||
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
|
||||
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
|
||||
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
|
||||
github.com/go-sql-driver/mysql v1.7.1 h1:lUIinVbN1DY0xBg0eMOzmmtGoHwWBbvnWubQUrtU8EI=
|
||||
github.com/go-sql-driver/mysql v1.7.1/go.mod h1:OXbVy3sEdcQ2Doequ6Z5BW6fXNQTmx+9S1MCJN5yJMI=
|
||||
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
|
||||
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
|
||||
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
|
||||
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
|
||||
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
|
||||
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
|
||||
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
|
||||
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
|
||||
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
|
||||
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
|
||||
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
|
||||
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
|
||||
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
|
||||
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
|
||||
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
|
||||
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 h1:RWengNIwukTxcDr9M+97sNutRR1RKhG96O6jWumTTnw=
|
||||
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826/go.mod h1:TaXosZuwdSHYgviHp1DAtfrULt5eUgsSMsZf+YrPgl8=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
|
||||
github.com/richardlehane/mscfb v1.0.4 h1:WULscsljNPConisD5hR0+OyZjwK46Pfyr6mPu5ZawpM=
|
||||
github.com/richardlehane/mscfb v1.0.4/go.mod h1:YzVpcZg9czvAuhk9T+a3avCpcFPMUWm7gK3DypaEsUk=
|
||||
github.com/richardlehane/msoleps v1.0.1/go.mod h1:BWev5JBpU9Ko2WAgmZEuiz4/u3ZYTKbjLycmwiWUfWg=
|
||||
github.com/richardlehane/msoleps v1.0.3 h1:aznSZzrwYRl3rLKRT3gUk9am7T/mLNSnJINvN0AQoVM=
|
||||
github.com/richardlehane/msoleps v1.0.3/go.mod h1:BWev5JBpU9Ko2WAgmZEuiz4/u3ZYTKbjLycmwiWUfWg=
|
||||
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
|
||||
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
|
||||
github.com/tjfoc/gmsm v1.4.1 h1:aMe1GlZb+0bLjn+cKTPEvvn9oUEBlJitaZiiBwsbgho=
|
||||
github.com/tjfoc/gmsm v1.4.1/go.mod h1:j4INPkHWMrhJb38G+J6W4Tw0AbuN8Thu3PbdVYhVcTE=
|
||||
github.com/xuri/efp v0.0.0-20231025114914-d1ff6096ae53 h1:Chd9DkqERQQuHpXjR/HSV1jLZA6uaoiwwH3vSuF3IW0=
|
||||
github.com/xuri/efp v0.0.0-20231025114914-d1ff6096ae53/go.mod h1:ybY/Jr0T0GTCnYjKqmdwxyxn2BQf2RcQIIvex5QldPI=
|
||||
github.com/xuri/excelize/v2 v2.8.1 h1:pZLMEwK8ep+CLIUWpWmvW8IWE/yxqG0I1xcN6cVMGuQ=
|
||||
github.com/xuri/excelize/v2 v2.8.1/go.mod h1:oli1E4C3Pa5RXg1TBXn4ENCXDV5JUMlBluUhG7c+CEE=
|
||||
github.com/xuri/nfp v0.0.0-20230919160717-d98342af3f05 h1:qhbILQo1K3mphbwKh1vNm4oGezE1eF9fQWmNiIpSfI4=
|
||||
github.com/xuri/nfp v0.0.0-20230919160717-d98342af3f05/go.mod h1:WwHg+CVyzlv/TX9xqBFXEZAuxOPxn2k1GNHwG41IIUQ=
|
||||
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
|
||||
golang.org/x/crypto v0.0.0-20201012173705-84dcc777aaee/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
|
||||
golang.org/x/crypto v0.19.0 h1:ENy+Az/9Y1vSrlrvBSyna3PITt4tiZLf7sgCjZBX7Wo=
|
||||
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
|
||||
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
|
||||
golang.org/x/image v0.14.0 h1:tNgSxAFe3jC4uYqvZdTr84SZoM1KfwdC9SKIFrLjFn4=
|
||||
golang.org/x/image v0.14.0/go.mod h1:HUYqC05R2ZcZ3ejNQsIHQDQiwWM4JBqmm6MKANTp4LE=
|
||||
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
|
||||
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
|
||||
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
|
||||
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
|
||||
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
|
||||
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
|
||||
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
|
||||
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
|
||||
golang.org/x/net v0.0.0-20201010224723-4f7140c49acb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
|
||||
golang.org/x/net v0.21.0 h1:AQyQV4dYCvJ7vGmJyKki9+PBdyvhkSd8EIx/qb0AYv4=
|
||||
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
|
||||
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
|
||||
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
|
||||
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
|
||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
|
||||
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
|
||||
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
|
||||
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
|
||||
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
|
||||
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
|
||||
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
|
||||
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
|
||||
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
|
||||
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
|
||||
google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
|
||||
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
|
||||
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
|
||||
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
|
||||
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
|
||||
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
|
||||
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
|
||||
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
|
||||
|
|
@ -0,0 +1,52 @@
|
|||
package api
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"time"
|
||||
"server/internal/logging"
|
||||
)
|
||||
|
||||
type statusWriter struct{
|
||||
http.ResponseWriter
|
||||
status int
|
||||
bytes int
|
||||
}
|
||||
|
||||
func (w *statusWriter) WriteHeader(code int){
|
||||
w.status = code
|
||||
w.ResponseWriter.WriteHeader(code)
|
||||
}
|
||||
|
||||
func (w *statusWriter) Write(b []byte)(int, error){
|
||||
n, err := w.ResponseWriter.Write(b)
|
||||
w.bytes += n
|
||||
return n, err
|
||||
}
|
||||
|
||||
func withAccess(h http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request){
|
||||
w.Header().Set("Access-Control-Allow-Origin", "*")
|
||||
w.Header().Set("Access-Control-Allow-Methods", "GET,POST,PATCH,DELETE,OPTIONS")
|
||||
w.Header().Set("Access-Control-Allow-Headers", "Content-Type, Authorization")
|
||||
if r.Method == http.MethodOptions {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
return
|
||||
}
|
||||
start := time.Now()
|
||||
sw := &statusWriter{ResponseWriter: w, status: 200}
|
||||
h.ServeHTTP(sw, r)
|
||||
dur := time.Since(start)
|
||||
m := MetaFrom(r)
|
||||
logging.JSON("INFO", map[string]interface{}{
|
||||
"kind": "access",
|
||||
"trace_id": TraceIDFrom(r),
|
||||
"method": m.Method,
|
||||
"path": m.Path,
|
||||
"query": m.Query,
|
||||
"remote": m.Remote,
|
||||
"status": sw.status,
|
||||
"bytes": sw.bytes,
|
||||
"duration_ms": dur.Milliseconds(),
|
||||
})
|
||||
})
|
||||
}
|
||||
|
|
@ -0,0 +1,105 @@
|
|||
package api
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type CreatorsAPI struct {
|
||||
marketing *sql.DB
|
||||
}
|
||||
|
||||
func CreatorsHandler(marketing *sql.DB) http.Handler {
|
||||
api := &CreatorsAPI{marketing: marketing}
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
p := strings.TrimPrefix(r.URL.Path, "/api/creators")
|
||||
if r.Method == http.MethodGet && p == "" {
|
||||
api.list(w, r)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
})
|
||||
}
|
||||
|
||||
func (a *CreatorsAPI) list(w http.ResponseWriter, r *http.Request) {
|
||||
q := r.URL.Query().Get("q")
|
||||
limitStr := r.URL.Query().Get("limit")
|
||||
limit := 2000
|
||||
if limitStr != "" {
|
||||
if n, err := strconv.Atoi(limitStr); err == nil && n > 0 && n <= 10000 { limit = n }
|
||||
}
|
||||
sql1 := "SELECT DISTINCT user_id, COALESCE(user_name, '') AS name FROM activity WHERE user_id IS NOT NULL"
|
||||
args := []interface{}{}
|
||||
if q != "" {
|
||||
sql1 += " AND (CAST(user_id AS CHAR) LIKE ? OR user_name LIKE ?)"
|
||||
like := "%" + q + "%"
|
||||
args = append(args, like, like)
|
||||
}
|
||||
sql1 += " ORDER BY user_id ASC LIMIT ?"
|
||||
args = append(args, limit)
|
||||
rows, err := a.marketing.Query(sql1, args...)
|
||||
out := []map[string]interface{}{}
|
||||
if err == nil {
|
||||
defer rows.Close()
|
||||
for rows.Next() {
|
||||
var id sql.NullInt64
|
||||
var name sql.NullString
|
||||
if err := rows.Scan(&id, &name); err != nil { continue }
|
||||
if !id.Valid { continue }
|
||||
m := map[string]interface{}{"id": id.Int64, "name": name.String}
|
||||
out = append(out, m)
|
||||
}
|
||||
}
|
||||
if err != nil || len(out) == 0 {
|
||||
sqlPlan := "SELECT DISTINCT creator, COALESCE(creator_name, '') AS name FROM plan WHERE creator IS NOT NULL"
|
||||
argsPlan := []interface{}{}
|
||||
if q != "" {
|
||||
sqlPlan += " AND (CAST(creator AS CHAR) LIKE ? OR creator_name LIKE ?)"
|
||||
like := "%" + q + "%"
|
||||
argsPlan = append(argsPlan, like, like)
|
||||
}
|
||||
sqlPlan += " ORDER BY creator ASC LIMIT ?"
|
||||
argsPlan = append(argsPlan, limit)
|
||||
rowsPlan, errPlan := a.marketing.Query(sqlPlan, argsPlan...)
|
||||
if errPlan == nil {
|
||||
defer rowsPlan.Close()
|
||||
tmp := []map[string]interface{}{}
|
||||
for rowsPlan.Next() {
|
||||
var id sql.NullInt64
|
||||
var name sql.NullString
|
||||
if err := rowsPlan.Scan(&id, &name); err != nil { continue }
|
||||
if !id.Valid { continue }
|
||||
tmp = append(tmp, map[string]interface{}{"id": id.Int64, "name": name.String})
|
||||
}
|
||||
if len(tmp) > 0 { out = tmp }
|
||||
}
|
||||
if len(out) == 0 {
|
||||
sql2 := "SELECT DISTINCT creator, '' AS name FROM `order` WHERE creator IS NOT NULL"
|
||||
args2 := []interface{}{}
|
||||
if q != "" {
|
||||
sql2 += " AND CAST(creator AS CHAR) LIKE ?"
|
||||
args2 = append(args2, "%"+q+"%")
|
||||
}
|
||||
sql2 += " ORDER BY creator ASC LIMIT ?"
|
||||
args2 = append(args2, limit)
|
||||
rows2, err2 := a.marketing.Query(sql2, args2...)
|
||||
if err2 != nil {
|
||||
fail(w, r, http.StatusInternalServerError, err2.Error())
|
||||
return
|
||||
}
|
||||
defer rows2.Close()
|
||||
out = out[:0]
|
||||
for rows2.Next() {
|
||||
var id sql.NullInt64
|
||||
var name sql.NullString
|
||||
if err := rows2.Scan(&id, &name); err != nil { continue }
|
||||
if !id.Valid { continue }
|
||||
m := map[string]interface{}{"id": id.Int64, "name": name.String}
|
||||
out = append(out, m)
|
||||
}
|
||||
}
|
||||
}
|
||||
ok(w, r, out)
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,167 @@
|
|||
package api
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"net/http"
|
||||
"sort"
|
||||
)
|
||||
|
||||
func MetadataHandler(meta, marketing, ymt *sql.DB) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
ds := r.URL.Query().Get("datasource")
|
||||
ot := r.URL.Query().Get("order_type")
|
||||
db := marketing
|
||||
if ds == "ymt" {
|
||||
db = ymt
|
||||
}
|
||||
tables := []string{}
|
||||
if ds == "ymt" {
|
||||
tables = []string{"order_info", "order_cash", "order_voucher", "order_digit", "goods_voucher_batch", "goods_voucher_subject_config", "merchant", "activity"}
|
||||
} else {
|
||||
tables = []string{"order", "order_detail", "order_cash", "order_voucher", "plan", "key_batch", "code_batch", "voucher", "voucher_batch", "merchant_key_send"}
|
||||
}
|
||||
out := []map[string]interface{}{}
|
||||
for _, tbl := range tables {
|
||||
cols := getColumns(db, tbl)
|
||||
fields := []map[string]string{}
|
||||
for _, c := range cols {
|
||||
tCanonical, fCanonical := canonicalField(ds, tbl, c.Name)
|
||||
if tCanonical == "" || fCanonical == "" {
|
||||
continue
|
||||
}
|
||||
lab := c.Comment
|
||||
if lab == "" {
|
||||
lab = fCanonical
|
||||
}
|
||||
fields = append(fields, map[string]string{"key": tCanonical + "." + fCanonical, "field": fCanonical, "label": lab})
|
||||
}
|
||||
tDisplay := displayTable(ds, tbl)
|
||||
out = append(out, map[string]interface{}{"table": tDisplay, "label": tableLabel(tDisplay), "fields": fields})
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool { return out[i]["table"].(string) < out[j]["table"].(string) })
|
||||
rec := recommendedDefaults(ds, ot)
|
||||
ok(w, r, map[string]interface{}{"datasource": ds, "tables": out, "recommended": rec})
|
||||
})
|
||||
}
|
||||
|
||||
func tableLabel(t string) string {
|
||||
switch t {
|
||||
case "order":
|
||||
return "订单主表"
|
||||
case "order_detail":
|
||||
return "订单详情"
|
||||
case "order_cash":
|
||||
return "红包订单"
|
||||
case "order_voucher":
|
||||
return "立减金订单"
|
||||
case "order_digit":
|
||||
return "直充卡密订单"
|
||||
case "plan":
|
||||
return "活动"
|
||||
case "key_batch":
|
||||
return "key批次"
|
||||
case "code_batch":
|
||||
return "兑换码批次"
|
||||
case "voucher":
|
||||
return "立减金"
|
||||
case "voucher_batch":
|
||||
return "立减金批次"
|
||||
case "merchant_key_send":
|
||||
return "key码API发放记录"
|
||||
case "goods_voucher_batch":
|
||||
return "立减金批次表"
|
||||
case "goods_voucher_subject_config":
|
||||
return "立减金主体"
|
||||
case "merchant":
|
||||
return "客户"
|
||||
case "activity":
|
||||
return "活动"
|
||||
default:
|
||||
return t
|
||||
}
|
||||
}
|
||||
|
||||
func displayTable(ds, tbl string) string {
|
||||
if ds == "ymt" && tbl == "order_info" {
|
||||
return "order"
|
||||
}
|
||||
return tbl
|
||||
}
|
||||
|
||||
func canonicalField(ds, tbl, col string) (string, string) {
|
||||
if ds == "ymt" && tbl == "order_info" {
|
||||
switch col {
|
||||
case "order_no":
|
||||
return "order", "order_number"
|
||||
case "key_code":
|
||||
return "order", "key"
|
||||
case "user_id":
|
||||
return "order", "creator"
|
||||
case "out_order_no":
|
||||
return "order", "out_trade_no"
|
||||
case "activity_id":
|
||||
return "order", "plan_id"
|
||||
case "merchant_id":
|
||||
return "order", "reseller_id"
|
||||
case "goods_id":
|
||||
return "order", "product_id"
|
||||
case "pay_price":
|
||||
return "order", "pay_amount"
|
||||
case "key_batch_name":
|
||||
return "order", "key_batch_id"
|
||||
default:
|
||||
return "order", col
|
||||
}
|
||||
}
|
||||
// other tables: canonical equals actual
|
||||
return tbl, col
|
||||
}
|
||||
|
||||
type columnMeta struct {
|
||||
Name string
|
||||
Comment string
|
||||
}
|
||||
|
||||
func getColumns(db *sql.DB, tbl string) []columnMeta {
|
||||
rows, err := db.Query("SELECT COLUMN_NAME, COLUMN_COMMENT FROM information_schema.columns WHERE table_schema = DATABASE() AND table_name = ? ORDER BY ORDINAL_POSITION", tbl)
|
||||
if err != nil {
|
||||
return []columnMeta{}
|
||||
}
|
||||
defer rows.Close()
|
||||
cols := []columnMeta{}
|
||||
for rows.Next() {
|
||||
var name, comment string
|
||||
if err := rows.Scan(&name, &comment); err == nil {
|
||||
cols = append(cols, columnMeta{Name: name, Comment: comment})
|
||||
}
|
||||
}
|
||||
return cols
|
||||
}
|
||||
|
||||
func recommendedDefaults(ds, orderType string) []string {
|
||||
base := []string{"order.order_number", "order.creator", "order.out_trade_no", "order.type", "order.status", "order.contract_price", "order.num", "order.pay_amount", "order.create_time"}
|
||||
if ds != "ymt" {
|
||||
base = []string{"order.order_number", "order.creator", "order.out_trade_no", "order.type", "order.status", "order.contract_price", "order.num", "order.total", "order.pay_amount", "order.create_time"}
|
||||
}
|
||||
t := orderType
|
||||
if t == "2" { // 直充卡密
|
||||
if ds == "ymt" {
|
||||
base = append(base, "order_digit.order_no", "order_digit.account", "order_digit.success_time")
|
||||
} else {
|
||||
base = append(base, "plan.title")
|
||||
}
|
||||
} else if t == "3" { // 立减金
|
||||
if ds == "ymt" {
|
||||
base = append(base, "order_voucher.channel", "order_voucher.status", "goods_voucher_batch.channel_batch_no")
|
||||
} else {
|
||||
base = append(base, "order_voucher.channel", "voucher.denomination")
|
||||
}
|
||||
} else if t == "1" { // 红包
|
||||
if ds == "ymt" {
|
||||
base = append(base, "order_cash.channel", "order_cash.receive_status", "order_cash.denomination")
|
||||
} else {
|
||||
base = append(base, "order_cash.channel", "order_cash.receive_status", "order_cash.amount")
|
||||
}
|
||||
}
|
||||
return base
|
||||
}
|
||||
|
|
@ -0,0 +1,84 @@
|
|||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
)
|
||||
|
||||
type ctxKey string
|
||||
|
||||
var traceKey ctxKey = "trace_id"
|
||||
var sqlKey ctxKey = "sql"
|
||||
var metaKey ctxKey = "req_meta"
|
||||
var payloadKey ctxKey = "payload"
|
||||
|
||||
func withTrace(h http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
tid := r.Header.Get("X-Request-ID")
|
||||
if tid == "" {
|
||||
buf := make([]byte, 16)
|
||||
_, _ = rand.Read(buf)
|
||||
tid = hex.EncodeToString(buf)
|
||||
}
|
||||
w.Header().Set("X-Request-ID", tid)
|
||||
m := ReqMeta{Method: r.Method, Path: r.URL.Path, Query: r.URL.RawQuery, Remote: r.RemoteAddr}
|
||||
ctx := context.WithValue(r.Context(), traceKey, tid)
|
||||
ctx = context.WithValue(ctx, metaKey, m)
|
||||
h.ServeHTTP(w, r.WithContext(ctx))
|
||||
})
|
||||
}
|
||||
|
||||
func TraceIDFrom(r *http.Request) string {
|
||||
v := r.Context().Value(traceKey)
|
||||
if v == nil {
|
||||
return ""
|
||||
}
|
||||
s, _ := v.(string)
|
||||
return s
|
||||
}
|
||||
|
||||
func WithSQL(r *http.Request, sql string) *http.Request {
|
||||
return r.WithContext(context.WithValue(r.Context(), sqlKey, sql))
|
||||
}
|
||||
|
||||
func SQLFrom(r *http.Request) string {
|
||||
v := r.Context().Value(sqlKey)
|
||||
if v == nil {
|
||||
return ""
|
||||
}
|
||||
s, _ := v.(string)
|
||||
return s
|
||||
}
|
||||
|
||||
type ReqMeta struct {
|
||||
Method string
|
||||
Path string
|
||||
Query string
|
||||
Remote string
|
||||
}
|
||||
|
||||
func MetaFrom(r *http.Request) ReqMeta {
|
||||
v := r.Context().Value(metaKey)
|
||||
if v == nil {
|
||||
return ReqMeta{}
|
||||
}
|
||||
m, _ := v.(ReqMeta)
|
||||
return m
|
||||
}
|
||||
|
||||
func WithPayload(r *http.Request, v interface{}) *http.Request {
|
||||
b, _ := json.Marshal(v)
|
||||
return r.WithContext(context.WithValue(r.Context(), payloadKey, string(b)))
|
||||
}
|
||||
|
||||
func PayloadFrom(r *http.Request) string {
|
||||
v := r.Context().Value(payloadKey)
|
||||
if v == nil {
|
||||
return ""
|
||||
}
|
||||
s, _ := v.(string)
|
||||
return s
|
||||
}
|
||||
|
|
@ -0,0 +1,75 @@
|
|||
package api
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type PlansAPI struct {
|
||||
marketing *sql.DB
|
||||
}
|
||||
|
||||
func PlansHandler(marketing *sql.DB) http.Handler {
|
||||
api := &PlansAPI{marketing: marketing}
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
p := strings.TrimPrefix(r.URL.Path, "/api/plans")
|
||||
if r.Method == http.MethodGet && p == "" {
|
||||
api.list(w, r)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
})
|
||||
}
|
||||
|
||||
func (a *PlansAPI) list(w http.ResponseWriter, r *http.Request) {
|
||||
resellerParam := r.URL.Query().Get("reseller")
|
||||
if resellerParam == "" {
|
||||
resellerParam = r.URL.Query().Get("reseller_id")
|
||||
}
|
||||
q := r.URL.Query().Get("q")
|
||||
limitStr := r.URL.Query().Get("limit")
|
||||
limit := 2000
|
||||
if limitStr != "" {
|
||||
if n, err := strconv.Atoi(limitStr); err == nil && n > 0 && n <= 10000 { limit = n }
|
||||
}
|
||||
resellers := []string{}
|
||||
for _, s := range strings.Split(resellerParam, ",") {
|
||||
s = strings.TrimSpace(s)
|
||||
if s != "" { resellers = append(resellers, s) }
|
||||
}
|
||||
if len(resellers) == 0 {
|
||||
ok(w, r, []map[string]interface{}{})
|
||||
return
|
||||
}
|
||||
ph := strings.Repeat("?,", len(resellers))
|
||||
ph = strings.TrimSuffix(ph, ",")
|
||||
sql1 := "SELECT id, COALESCE(title,'') AS title FROM plan WHERE reseller_id IN (" + ph + ")"
|
||||
args := []interface{}{}
|
||||
for _, v := range resellers { args = append(args, v) }
|
||||
if q != "" {
|
||||
sql1 += " AND (CAST(id AS CHAR) LIKE ? OR title LIKE ?)"
|
||||
like := "%" + q + "%"
|
||||
args = append(args, like, like)
|
||||
}
|
||||
sql1 += " ORDER BY id ASC LIMIT ?"
|
||||
args = append(args, limit)
|
||||
rows, err := a.marketing.Query(sql1, args...)
|
||||
if err != nil {
|
||||
fail(w, r, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []map[string]interface{}{}
|
||||
for rows.Next() {
|
||||
var id sql.NullInt64
|
||||
var title sql.NullString
|
||||
if err := rows.Scan(&id, &title); err != nil { continue }
|
||||
if !id.Valid { continue }
|
||||
m := map[string]interface{}{"id": id.Int64, "title": title.String}
|
||||
out = append(out, m)
|
||||
}
|
||||
ok(w, r, out)
|
||||
}
|
||||
|
||||
|
|
@ -0,0 +1,72 @@
|
|||
package api
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type ResellersAPI struct {
|
||||
marketing *sql.DB
|
||||
}
|
||||
|
||||
func ResellersHandler(marketing *sql.DB) http.Handler {
|
||||
api := &ResellersAPI{marketing: marketing}
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
p := strings.TrimPrefix(r.URL.Path, "/api/resellers")
|
||||
if r.Method == http.MethodGet && p == "" {
|
||||
api.list(w, r)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
})
|
||||
}
|
||||
|
||||
func (a *ResellersAPI) list(w http.ResponseWriter, r *http.Request) {
|
||||
creatorsParam := r.URL.Query().Get("creator")
|
||||
q := r.URL.Query().Get("q")
|
||||
limitStr := r.URL.Query().Get("limit")
|
||||
limit := 2000
|
||||
if limitStr != "" {
|
||||
if n, err := strconv.Atoi(limitStr); err == nil && n > 0 && n <= 10000 { limit = n }
|
||||
}
|
||||
creators := []string{}
|
||||
for _, s := range strings.Split(creatorsParam, ",") {
|
||||
s = strings.TrimSpace(s)
|
||||
if s != "" { creators = append(creators, s) }
|
||||
}
|
||||
if len(creators) == 0 {
|
||||
ok(w, r, []map[string]interface{}{})
|
||||
return
|
||||
}
|
||||
ph := strings.Repeat("?,", len(creators))
|
||||
ph = strings.TrimSuffix(ph, ",")
|
||||
sql1 := "SELECT DISTINCT reseller_id, COALESCE(reseller_name,'') AS name FROM plan WHERE reseller_id IS NOT NULL AND creator IN (" + ph + ")"
|
||||
args := []interface{}{}
|
||||
for _, c := range creators { args = append(args, c) }
|
||||
if q != "" {
|
||||
sql1 += " AND (CAST(reseller_id AS CHAR) LIKE ? OR reseller_name LIKE ?)"
|
||||
like := "%" + q + "%"
|
||||
args = append(args, like, like)
|
||||
}
|
||||
sql1 += " ORDER BY reseller_id ASC LIMIT ?"
|
||||
args = append(args, limit)
|
||||
rows, err := a.marketing.Query(sql1, args...)
|
||||
if err != nil {
|
||||
fail(w, r, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []map[string]interface{}{}
|
||||
for rows.Next() {
|
||||
var id sql.NullInt64
|
||||
var name sql.NullString
|
||||
if err := rows.Scan(&id, &name); err != nil { continue }
|
||||
if !id.Valid { continue }
|
||||
m := map[string]interface{}{"id": id.Int64, "name": name.String}
|
||||
out = append(out, m)
|
||||
}
|
||||
ok(w, r, out)
|
||||
}
|
||||
|
||||
|
|
@ -0,0 +1,51 @@
|
|||
package api
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"log"
|
||||
"net/http"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
)
|
||||
|
||||
type resp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data interface{} `json:"data"`
|
||||
TraceID string `json:"trace_id"`
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, r *http.Request, status int, code int, msg string, data interface{}) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
tid := TraceIDFrom(r)
|
||||
b, _ := json.Marshal(resp{Code: code, Msg: msg, Data: data, TraceID: tid})
|
||||
w.Write(b)
|
||||
if code != 0 {
|
||||
_, file, line, _ := runtime.Caller(1)
|
||||
base := filepath.Base(file)
|
||||
sql := SQLFrom(r)
|
||||
meta := MetaFrom(r)
|
||||
payload := PayloadFrom(r)
|
||||
if sql != "" {
|
||||
log.Printf("trace_id=%s status=%d file=%s:%d method=%s path=%s query=%s remote=%s sql=%s payload=%s msg=%s", tid, status, base, line, meta.Method, meta.Path, meta.Query, meta.Remote, sql, payload, msg)
|
||||
} else {
|
||||
log.Printf("trace_id=%s status=%d file=%s:%d method=%s path=%s query=%s remote=%s payload=%s msg=%s", tid, status, base, line, meta.Method, meta.Path, meta.Query, meta.Remote, payload, msg)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func ok(w http.ResponseWriter, r *http.Request, data interface{}) {
|
||||
writeJSON(w, r, http.StatusOK, 0, "ok", data)
|
||||
}
|
||||
|
||||
func fail(w http.ResponseWriter, r *http.Request, status int, msg string) {
|
||||
writeJSON(w, r, status, 1, msg, nil)
|
||||
}
|
||||
|
||||
func failCat(w http.ResponseWriter, r *http.Request, status int, msg string, kind string) {
|
||||
writeJSON(w, r, status, 1, msg, nil)
|
||||
tid := TraceIDFrom(r)
|
||||
meta := MetaFrom(r)
|
||||
log.Printf("kind=%s trace_id=%s status=%d method=%s path=%s msg=%s", kind, tid, status, meta.Method, meta.Path, msg)
|
||||
}
|
||||
|
|
@ -0,0 +1,52 @@
|
|||
package api
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"net/http"
|
||||
"os"
|
||||
)
|
||||
|
||||
func NewRouter(metaDB *sql.DB, marketingDB *sql.DB, ymtDB *sql.DB) http.Handler {
|
||||
mux := http.NewServeMux()
|
||||
mux.Handle("/api/templates", withAccess(withTrace(TemplatesHandler(metaDB, marketingDB))))
|
||||
mux.Handle("/api/templates/", withAccess(withTrace(TemplatesHandler(metaDB, marketingDB))))
|
||||
mux.Handle("/api/exports", withAccess(withTrace(ExportsHandler(metaDB, marketingDB, ymtDB))))
|
||||
mux.Handle("/api/exports/", withAccess(withTrace(ExportsHandler(metaDB, marketingDB, ymtDB))))
|
||||
mux.Handle("/api/metadata/fields", withAccess(withTrace(MetadataHandler(metaDB, marketingDB, ymtDB))))
|
||||
mux.Handle("/api/creators", withAccess(withTrace(CreatorsHandler(marketingDB))))
|
||||
mux.Handle("/api/creators/", withAccess(withTrace(CreatorsHandler(marketingDB))))
|
||||
mux.Handle("/api/resellers", withAccess(withTrace(ResellersHandler(marketingDB))))
|
||||
mux.Handle("/api/resellers/", withAccess(withTrace(ResellersHandler(marketingDB))))
|
||||
mux.Handle("/api/plans", withAccess(withTrace(PlansHandler(marketingDB))))
|
||||
mux.Handle("/api/plans/", withAccess(withTrace(PlansHandler(marketingDB))))
|
||||
mux.Handle("/api/ymt/users", withAccess(withTrace(YMTUsersHandler(ymtDB))))
|
||||
mux.Handle("/api/ymt/users/", withAccess(withTrace(YMTUsersHandler(ymtDB))))
|
||||
mux.Handle("/api/ymt/merchants", withAccess(withTrace(YMTMerchantsHandler(ymtDB))))
|
||||
mux.Handle("/api/ymt/merchants/", withAccess(withTrace(YMTMerchantsHandler(ymtDB))))
|
||||
mux.Handle("/api/ymt/activities", withAccess(withTrace(YMTActivitiesHandler(ymtDB))))
|
||||
mux.Handle("/api/ymt/activities/", withAccess(withTrace(YMTActivitiesHandler(ymtDB))))
|
||||
mux.HandleFunc("/api/utils/decode_key", func(w http.ResponseWriter, r *http.Request) {
|
||||
v := r.URL.Query().Get("v")
|
||||
if v == "" {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
w.Write([]byte("missing v"))
|
||||
return
|
||||
}
|
||||
d := decodeOrderKey(v)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.Write([]byte("{\"decoded\":\"" + d + "\"}"))
|
||||
})
|
||||
sd := staticDir()
|
||||
mux.Handle("/", http.FileServer(http.Dir(sd)))
|
||||
return mux
|
||||
}
|
||||
|
||||
func staticDir() string {
|
||||
if _, err := os.Stat("web/index.html"); err == nil {
|
||||
return "web"
|
||||
}
|
||||
if _, err := os.Stat("../web/index.html"); err == nil {
|
||||
return "../web"
|
||||
}
|
||||
return "web"
|
||||
}
|
||||
|
|
@ -0,0 +1,328 @@
|
|||
package api
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"server/internal/exporter"
|
||||
"server/internal/schema"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
type TemplatesAPI struct {
|
||||
meta *sql.DB
|
||||
marketing *sql.DB
|
||||
}
|
||||
|
||||
func TemplatesHandler(meta, marketing *sql.DB) http.Handler {
|
||||
api := &TemplatesAPI{meta: meta, marketing: marketing}
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
p := strings.TrimPrefix(r.URL.Path, "/api/templates")
|
||||
if r.Method == http.MethodPost && p == "" {
|
||||
api.createTemplate(w, r)
|
||||
return
|
||||
}
|
||||
if r.Method == http.MethodGet && p == "" {
|
||||
api.listTemplates(w, r)
|
||||
return
|
||||
}
|
||||
if strings.HasPrefix(p, "/") {
|
||||
id := strings.TrimPrefix(p, "/")
|
||||
if r.Method == http.MethodGet {
|
||||
api.getTemplate(w, r, id)
|
||||
return
|
||||
}
|
||||
if r.Method == http.MethodPatch {
|
||||
api.patchTemplate(w, r, id)
|
||||
return
|
||||
}
|
||||
if r.Method == http.MethodDelete {
|
||||
api.deleteTemplate(w, r, id)
|
||||
return
|
||||
}
|
||||
if r.Method == http.MethodPost && strings.HasSuffix(p, "/validate") {
|
||||
id = strings.TrimSuffix(id, "/validate")
|
||||
api.validateTemplate(w, r, id)
|
||||
return
|
||||
}
|
||||
}
|
||||
fail(w, r, http.StatusNotFound, "not found")
|
||||
})
|
||||
}
|
||||
|
||||
type TemplatePayload struct {
|
||||
Name string `json:"name"`
|
||||
Datasource string `json:"datasource"`
|
||||
MainTable string `json:"main_table"`
|
||||
Fields []string `json:"fields"`
|
||||
Filters map[string]interface{} `json:"filters"`
|
||||
FileFormat string `json:"file_format"`
|
||||
OwnerID uint64 `json:"owner_id"`
|
||||
Visibility string `json:"visibility"`
|
||||
}
|
||||
|
||||
func (a *TemplatesAPI) createTemplate(w http.ResponseWriter, r *http.Request) {
|
||||
b, _ := io.ReadAll(r.Body)
|
||||
var p TemplatePayload
|
||||
json.Unmarshal(b, &p)
|
||||
r = WithPayload(r, p)
|
||||
uidStr := r.URL.Query().Get("userId")
|
||||
if uidStr != "" {
|
||||
var uid uint64
|
||||
_, _ = fmt.Sscan(uidStr, &uid)
|
||||
if uid > 0 {
|
||||
p.OwnerID = uid
|
||||
}
|
||||
}
|
||||
now := time.Now()
|
||||
tplSQL := "INSERT INTO export_templates (name, datasource, main_table, fields_json, filters_json, file_format, visibility, owner_id, enabled, stats_enabled, last_validated_at, created_at, updated_at) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)"
|
||||
tplArgs := []interface{}{p.Name, p.Datasource, p.MainTable, toJSON(p.Fields), toJSON(p.Filters), p.FileFormat, p.Visibility, p.OwnerID, 1, 0, now, now, now}
|
||||
log.Printf("trace_id=%s sql=%s args=%v", TraceIDFrom(r), tplSQL, tplArgs)
|
||||
_, err := a.meta.Exec(tplSQL, tplArgs...)
|
||||
if err != nil {
|
||||
fail(w, r, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, r, http.StatusCreated, 0, "ok", nil)
|
||||
}
|
||||
|
||||
func (a *TemplatesAPI) listTemplates(w http.ResponseWriter, r *http.Request) {
|
||||
uidStr := r.URL.Query().Get("userId")
|
||||
sqlText := "SELECT id,name,datasource,main_table,file_format,visibility,owner_id,enabled,last_validated_at,created_at,updated_at, COALESCE(JSON_LENGTH(fields_json),0) AS field_count, (SELECT COUNT(1) FROM export_jobs ej WHERE ej.template_id = export_templates.id) AS exec_count FROM export_templates"
|
||||
args := []interface{}{}
|
||||
conds := []string{}
|
||||
if uidStr != "" {
|
||||
conds = append(conds, "owner_id IN (0, ?)")
|
||||
args = append(args, uidStr)
|
||||
}
|
||||
conds = append(conds, "enabled = 1")
|
||||
if len(conds) > 0 {
|
||||
sqlText += " WHERE " + strings.Join(conds, " AND ")
|
||||
}
|
||||
sqlText += " ORDER BY datasource ASC, id DESC LIMIT 200"
|
||||
rows, err := a.meta.Query(sqlText, args...)
|
||||
if err != nil {
|
||||
fail(w, r, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []map[string]interface{}{}
|
||||
for rows.Next() {
|
||||
var id uint64
|
||||
var name, datasource, mainTable, fileFormat, visibility string
|
||||
var ownerID uint64
|
||||
var enabled int
|
||||
var lastValidatedAt sql.NullTime
|
||||
var createdAt, updatedAt time.Time
|
||||
var fieldCount, execCount int64
|
||||
err := rows.Scan(&id, &name, &datasource, &mainTable, &fileFormat, &visibility, &ownerID, &enabled, &lastValidatedAt, &createdAt, &updatedAt, &fieldCount, &execCount)
|
||||
if err != nil {
|
||||
fail(w, r, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
m := map[string]interface{}{"id": id, "name": name, "datasource": datasource, "main_table": mainTable, "file_format": fileFormat, "visibility": visibility, "owner_id": ownerID, "enabled": enabled == 1, "last_validated_at": lastValidatedAt.Time, "created_at": createdAt, "updated_at": updatedAt, "field_count": fieldCount, "exec_count": execCount}
|
||||
out = append(out, m)
|
||||
}
|
||||
ok(w, r, out)
|
||||
}
|
||||
|
||||
func (a *TemplatesAPI) getTemplate(w http.ResponseWriter, r *http.Request, id string) {
|
||||
row := a.meta.QueryRow("SELECT id,name,datasource,main_table,fields_json,filters_json,file_format,visibility,owner_id,enabled,explain_score,last_validated_at,created_at,updated_at FROM export_templates WHERE id=?", id)
|
||||
var m = map[string]interface{}{}
|
||||
var tid uint64
|
||||
var name, datasource, mainTable, fileFormat, visibility string
|
||||
var ownerID uint64
|
||||
var enabled int
|
||||
var explainScore sql.NullInt64
|
||||
var lastValidatedAt sql.NullTime
|
||||
var createdAt, updatedAt time.Time
|
||||
var fields, filters []byte
|
||||
err := row.Scan(&tid, &name, &datasource, &mainTable, &fields, &filters, &fileFormat, &visibility, &ownerID, &enabled, &explainScore, &lastValidatedAt, &createdAt, &updatedAt)
|
||||
if err != nil {
|
||||
fail(w, r, http.StatusNotFound, "not found")
|
||||
return
|
||||
}
|
||||
m["id"] = tid
|
||||
m["name"] = name
|
||||
m["datasource"] = datasource
|
||||
m["main_table"] = mainTable
|
||||
m["file_format"] = fileFormat
|
||||
m["visibility"] = visibility
|
||||
m["owner_id"] = ownerID
|
||||
m["enabled"] = enabled == 1
|
||||
m["explain_score"] = explainScore.Int64
|
||||
m["last_validated_at"] = lastValidatedAt.Time
|
||||
m["created_at"] = createdAt
|
||||
m["updated_at"] = updatedAt
|
||||
m["fields"] = fromJSON(fields)
|
||||
m["filters"] = fromJSON(filters)
|
||||
ok(w, r, m)
|
||||
}
|
||||
|
||||
func (a *TemplatesAPI) patchTemplate(w http.ResponseWriter, r *http.Request, id string) {
|
||||
b, err := io.ReadAll(r.Body)
|
||||
if err != nil {
|
||||
log.Printf("trace_id=%s error reading request body: %v", TraceIDFrom(r), err)
|
||||
fail(w, r, http.StatusBadRequest, "invalid request body")
|
||||
return
|
||||
}
|
||||
|
||||
log.Printf("trace_id=%s patchTemplate request body: %s", TraceIDFrom(r), string(b))
|
||||
|
||||
var p map[string]interface{}
|
||||
err = json.Unmarshal(b, &p)
|
||||
if err != nil {
|
||||
log.Printf("trace_id=%s error unmarshaling request body: %v", TraceIDFrom(r), err)
|
||||
fail(w, r, http.StatusBadRequest, "invalid JSON format")
|
||||
return
|
||||
}
|
||||
|
||||
log.Printf("trace_id=%s patchTemplate parsed payload: %v", TraceIDFrom(r), p)
|
||||
log.Printf("trace_id=%s patchTemplate template ID: %s", TraceIDFrom(r), id)
|
||||
|
||||
set := []string{}
|
||||
args := []interface{}{}
|
||||
for k, v := range p {
|
||||
log.Printf("trace_id=%s patchTemplate processing field: %s, value: %v, type: %T", TraceIDFrom(r), k, v, v)
|
||||
switch k {
|
||||
case "name", "visibility", "file_format", "main_table":
|
||||
if strVal, ok := v.(string); ok {
|
||||
set = append(set, k+"=?")
|
||||
args = append(args, strVal)
|
||||
log.Printf("trace_id=%s patchTemplate added string field: %s, value: %s", TraceIDFrom(r), k, strVal)
|
||||
} else {
|
||||
log.Printf("trace_id=%s patchTemplate invalid string field: %s, value: %v, type: %T", TraceIDFrom(r), k, v, v)
|
||||
}
|
||||
case "fields":
|
||||
set = append(set, "fields_json=?")
|
||||
jsonBytes := toJSON(v)
|
||||
args = append(args, jsonBytes)
|
||||
log.Printf("trace_id=%s patchTemplate added fields_json: %s", TraceIDFrom(r), string(jsonBytes))
|
||||
case "filters":
|
||||
set = append(set, "filters_json=?")
|
||||
jsonBytes := toJSON(v)
|
||||
args = append(args, jsonBytes)
|
||||
log.Printf("trace_id=%s patchTemplate added filters_json: %s", TraceIDFrom(r), string(jsonBytes))
|
||||
case "enabled":
|
||||
set = append(set, "enabled=?")
|
||||
if boolVal, ok := v.(bool); ok {
|
||||
if boolVal {
|
||||
args = append(args, 1)
|
||||
} else {
|
||||
args = append(args, 0)
|
||||
}
|
||||
log.Printf("trace_id=%s patchTemplate added enabled: %t", TraceIDFrom(r), boolVal)
|
||||
} else {
|
||||
log.Printf("trace_id=%s patchTemplate invalid bool field: %s, value: %v, type: %T", TraceIDFrom(r), k, v, v)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if len(set) == 0 {
|
||||
log.Printf("trace_id=%s patchTemplate no fields to update", TraceIDFrom(r))
|
||||
fail(w, r, http.StatusBadRequest, "no patch")
|
||||
return
|
||||
}
|
||||
|
||||
// ensure updated_at
|
||||
set = append(set, "updated_at=?")
|
||||
now := time.Now()
|
||||
args = append(args, now, id)
|
||||
|
||||
sql := "UPDATE export_templates SET " + strings.Join(set, ",") + " WHERE id= ?"
|
||||
log.Printf("trace_id=%s patchTemplate executing SQL: %s", TraceIDFrom(r), sql)
|
||||
log.Printf("trace_id=%s patchTemplate SQL args: %v", TraceIDFrom(r), args)
|
||||
|
||||
_, err = a.meta.Exec(sql, args...)
|
||||
if err != nil {
|
||||
log.Printf("trace_id=%s patchTemplate SQL error: %v", TraceIDFrom(r), err)
|
||||
fail(w, r, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
log.Printf("trace_id=%s patchTemplate update successful", TraceIDFrom(r))
|
||||
ok(w, r, nil)
|
||||
}
|
||||
|
||||
func (a *TemplatesAPI) deleteTemplate(w http.ResponseWriter, r *http.Request, id string) {
|
||||
var cnt int64
|
||||
row := a.meta.QueryRow("SELECT COUNT(1) FROM export_jobs WHERE template_id=?", id)
|
||||
_ = row.Scan(&cnt)
|
||||
if cnt > 0 {
|
||||
soft := strings.ToLower(strings.TrimSpace(r.URL.Query().Get("soft")))
|
||||
if soft == "1" || soft == "true" || soft == "yes" {
|
||||
a.meta.Exec("UPDATE export_templates SET enabled=?, updated_at=? WHERE id= ?", 0, time.Now(), id)
|
||||
ok(w, r, nil)
|
||||
return
|
||||
}
|
||||
fail(w, r, http.StatusBadRequest, "template in use")
|
||||
return
|
||||
}
|
||||
_, err := a.meta.Exec("DELETE FROM export_templates WHERE id= ?", id)
|
||||
if err != nil {
|
||||
fail(w, r, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
ok(w, r, nil)
|
||||
}
|
||||
|
||||
func (a *TemplatesAPI) validateTemplate(w http.ResponseWriter, r *http.Request, id string) {
|
||||
row := a.meta.QueryRow("SELECT datasource, main_table, fields_json, filters_json FROM export_templates WHERE id= ?", id)
|
||||
var ds string
|
||||
var main string
|
||||
var fields, filters []byte
|
||||
err := row.Scan(&ds, &main, &fields, &filters)
|
||||
if err != nil {
|
||||
fail(w, r, http.StatusNotFound, "not found")
|
||||
return
|
||||
}
|
||||
var fs []string
|
||||
var fl map[string]interface{}
|
||||
json.Unmarshal(fields, &fs)
|
||||
json.Unmarshal(filters, &fl)
|
||||
wl := Whitelist()
|
||||
req := exporter.BuildRequest{MainTable: main, Datasource: ds, Fields: fs, Filters: fl}
|
||||
q, args, err := exporter.BuildSQL(req, wl)
|
||||
if err != nil {
|
||||
failCat(w, r, http.StatusBadRequest, err.Error(), "sql_build_error")
|
||||
return
|
||||
}
|
||||
dataDB := a.selectDataDB(ds)
|
||||
score, sugg, err := exporter.EvaluateExplain(dataDB, q, args)
|
||||
if err != nil {
|
||||
failCat(w, r, http.StatusBadRequest, err.Error(), "explain_error")
|
||||
return
|
||||
}
|
||||
idxSugg := exporter.IndexSuggestions(req)
|
||||
sugg = append(sugg, idxSugg...)
|
||||
_, _ = a.meta.Exec("UPDATE export_templates SET explain_json=?, explain_score=?, last_validated_at=?, updated_at=? WHERE id=?", toJSON(map[string]interface{}{"sql": q, "suggestions": sugg}), score, time.Now(), time.Now(), id)
|
||||
ok(w, r, map[string]interface{}{"score": score, "suggestions": sugg})
|
||||
}
|
||||
|
||||
func (a *TemplatesAPI) selectDataDB(ds string) *sql.DB {
|
||||
if ds == "ymt" {
|
||||
return a.meta
|
||||
}
|
||||
return a.marketing
|
||||
}
|
||||
|
||||
func toJSON(v interface{}) []byte {
|
||||
b, _ := json.Marshal(v)
|
||||
return b
|
||||
}
|
||||
|
||||
func fromJSON(b []byte) interface{} {
|
||||
var v interface{}
|
||||
json.Unmarshal(b, &v)
|
||||
return v
|
||||
}
|
||||
|
||||
func Whitelist() map[string]bool { return schema.AllWhitelist() }
|
||||
|
||||
func FieldLabels() map[string]string {
|
||||
return schema.AllLabels()
|
||||
}
|
||||
|
|
@ -0,0 +1,62 @@
|
|||
package api
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"net/http"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
type YMTActivitiesAPI struct {
|
||||
ymt *sql.DB
|
||||
}
|
||||
|
||||
func YMTActivitiesHandler(ymt *sql.DB) http.Handler {
|
||||
api := &YMTActivitiesAPI{ymt: ymt}
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method == http.MethodGet {
|
||||
api.list(w, r)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
})
|
||||
}
|
||||
|
||||
func (a *YMTActivitiesAPI) list(w http.ResponseWriter, r *http.Request) {
|
||||
q := r.URL.Query()
|
||||
merchantIDStr := q.Get("merchant_id")
|
||||
like := q.Get("q")
|
||||
limitStr := q.Get("limit")
|
||||
limit := 2000
|
||||
if limitStr != "" {
|
||||
if n, err := strconv.Atoi(limitStr); err == nil && n > 0 && n <= 10000 { limit = n }
|
||||
}
|
||||
sql1 := "SELECT id, name FROM activity WHERE id IS NOT NULL"
|
||||
args := []interface{}{}
|
||||
if merchantIDStr != "" {
|
||||
sql1 += " AND merchant_id = ?"
|
||||
args = append(args, merchantIDStr)
|
||||
}
|
||||
if like != "" {
|
||||
sql1 += " AND (CAST(id AS CHAR) LIKE ? OR name LIKE ?)"
|
||||
s := "%" + like + "%"
|
||||
args = append(args, s, s)
|
||||
}
|
||||
sql1 += " ORDER BY id ASC LIMIT ?"
|
||||
args = append(args, limit)
|
||||
rows, err := a.ymt.Query(sql1, args...)
|
||||
if err != nil {
|
||||
fail(w, r, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []map[string]interface{}{}
|
||||
for rows.Next() {
|
||||
var id sql.NullInt64
|
||||
var name sql.NullString
|
||||
if err := rows.Scan(&id, &name); err != nil { continue }
|
||||
if !id.Valid { continue }
|
||||
out = append(out, map[string]interface{}{"id": id.Int64, "name": name.String})
|
||||
}
|
||||
ok(w, r, out)
|
||||
}
|
||||
|
||||
|
|
@ -0,0 +1,62 @@
|
|||
package api
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"net/http"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
type YMTMerchantsAPI struct {
|
||||
ymt *sql.DB
|
||||
}
|
||||
|
||||
func YMTMerchantsHandler(ymt *sql.DB) http.Handler {
|
||||
api := &YMTMerchantsAPI{ymt: ymt}
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method == http.MethodGet {
|
||||
api.list(w, r)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
})
|
||||
}
|
||||
|
||||
func (a *YMTMerchantsAPI) list(w http.ResponseWriter, r *http.Request) {
|
||||
q := r.URL.Query()
|
||||
userIDStr := q.Get("user_id")
|
||||
like := q.Get("q")
|
||||
limitStr := q.Get("limit")
|
||||
limit := 2000
|
||||
if limitStr != "" {
|
||||
if n, err := strconv.Atoi(limitStr); err == nil && n > 0 && n <= 10000 { limit = n }
|
||||
}
|
||||
sql1 := "SELECT id, name FROM merchant WHERE id IS NOT NULL"
|
||||
args := []interface{}{}
|
||||
if userIDStr != "" {
|
||||
sql1 += " AND user_id = ?"
|
||||
args = append(args, userIDStr)
|
||||
}
|
||||
if like != "" {
|
||||
sql1 += " AND (CAST(id AS CHAR) LIKE ? OR name LIKE ?)"
|
||||
s := "%" + like + "%"
|
||||
args = append(args, s, s)
|
||||
}
|
||||
sql1 += " ORDER BY id ASC LIMIT ?"
|
||||
args = append(args, limit)
|
||||
rows, err := a.ymt.Query(sql1, args...)
|
||||
if err != nil {
|
||||
fail(w, r, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []map[string]interface{}{}
|
||||
for rows.Next() {
|
||||
var id sql.NullInt64
|
||||
var name sql.NullString
|
||||
if err := rows.Scan(&id, &name); err != nil { continue }
|
||||
if !id.Valid { continue }
|
||||
out = append(out, map[string]interface{}{"id": id.Int64, "name": name.String})
|
||||
}
|
||||
ok(w, r, out)
|
||||
}
|
||||
|
||||
|
|
@ -0,0 +1,57 @@
|
|||
package api
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type YMTUsersAPI struct {
|
||||
ymt *sql.DB
|
||||
}
|
||||
|
||||
func YMTUsersHandler(ymt *sql.DB) http.Handler {
|
||||
api := &YMTUsersAPI{ymt: ymt}
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
p := strings.TrimPrefix(r.URL.Path, "/api/ymt/users")
|
||||
if r.Method == http.MethodGet && p == "" {
|
||||
api.list(w, r)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
})
|
||||
}
|
||||
|
||||
func (a *YMTUsersAPI) list(w http.ResponseWriter, r *http.Request) {
|
||||
q := r.URL.Query().Get("q")
|
||||
limitStr := r.URL.Query().Get("limit")
|
||||
limit := 2000
|
||||
if limitStr != "" {
|
||||
if n, err := strconv.Atoi(limitStr); err == nil && n > 0 && n <= 10000 { limit = n }
|
||||
}
|
||||
sql1 := "SELECT DISTINCT user_id, COALESCE(user_name, '') AS name FROM activity WHERE user_id IS NOT NULL"
|
||||
args := []interface{}{}
|
||||
if q != "" {
|
||||
sql1 += " AND (CAST(user_id AS CHAR) LIKE ? OR user_name LIKE ?)"
|
||||
like := "%" + q + "%"
|
||||
args = append(args, like, like)
|
||||
}
|
||||
sql1 += " ORDER BY user_id ASC LIMIT ?"
|
||||
args = append(args, limit)
|
||||
rows, err := a.ymt.Query(sql1, args...)
|
||||
if err != nil {
|
||||
fail(w, r, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []map[string]interface{}{}
|
||||
for rows.Next() {
|
||||
var id sql.NullInt64
|
||||
var name sql.NullString
|
||||
if err := rows.Scan(&id, &name); err != nil { continue }
|
||||
if !id.Valid { continue }
|
||||
out = append(out, map[string]interface{}{"id": id.Int64, "name": name.String})
|
||||
}
|
||||
ok(w, r, out)
|
||||
}
|
||||
|
|
@ -0,0 +1,152 @@
|
|||
package config
|
||||
|
||||
import (
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
type DB struct {
|
||||
Host string `yaml:"host"`
|
||||
Port string `yaml:"port"`
|
||||
User string `yaml:"user"`
|
||||
Password string `yaml:"password"`
|
||||
Name string `yaml:"name"`
|
||||
}
|
||||
|
||||
type App struct {
|
||||
Port string `yaml:"port"`
|
||||
MarketingDB DB `yaml:"marketing_db"`
|
||||
YMTDB DB `yaml:"ymt_db"`
|
||||
YMTTestDB DB `yaml:"ymt_test_db"`
|
||||
YMTKeyDecryptKeyB64 string `yaml:"ymt_key_decrypt_key_b64"`
|
||||
}
|
||||
|
||||
type root struct {
|
||||
App App `yaml:"app"`
|
||||
}
|
||||
|
||||
func Load() App {
|
||||
var cfg App
|
||||
// unified single config: prefer server/config.yaml, then config.yaml
|
||||
if !readYAML(filepath.Join("server", "config.yaml"), &cfg) {
|
||||
_ = readYAML("config.yaml", &cfg)
|
||||
}
|
||||
LoadEnv()
|
||||
if v := os.Getenv("MARKETING_DB_HOST"); v != "" {
|
||||
cfg.MarketingDB.Host = v
|
||||
}
|
||||
if v := os.Getenv("MARKETING_DB_PORT"); v != "" {
|
||||
cfg.MarketingDB.Port = v
|
||||
}
|
||||
if v := os.Getenv("MARKETING_DB_USER"); v != "" {
|
||||
cfg.MarketingDB.User = v
|
||||
}
|
||||
if v := os.Getenv("MARKETING_DB_PASSWORD"); v != "" {
|
||||
cfg.MarketingDB.Password = v
|
||||
}
|
||||
if v := os.Getenv("MARKETING_DB_NAME"); v != "" {
|
||||
cfg.MarketingDB.Name = v
|
||||
}
|
||||
if v := os.Getenv("YMT_DB_HOST"); v != "" {
|
||||
cfg.YMTDB.Host = v
|
||||
}
|
||||
if v := os.Getenv("YMT_DB_PORT"); v != "" {
|
||||
cfg.YMTDB.Port = v
|
||||
}
|
||||
if v := os.Getenv("YMT_DB_USER"); v != "" {
|
||||
cfg.YMTDB.User = v
|
||||
}
|
||||
if v := os.Getenv("YMT_DB_PASSWORD"); v != "" {
|
||||
cfg.YMTDB.Password = v
|
||||
}
|
||||
if v := os.Getenv("YMT_DB_NAME"); v != "" {
|
||||
cfg.YMTDB.Name = v
|
||||
}
|
||||
// optional env overrides for test DB
|
||||
if v := os.Getenv("YMT_TEST_DB_HOST"); v != "" {
|
||||
cfg.YMTTestDB.Host = v
|
||||
}
|
||||
if v := os.Getenv("YMT_TEST_DB_PORT"); v != "" {
|
||||
cfg.YMTTestDB.Port = v
|
||||
}
|
||||
if v := os.Getenv("YMT_TEST_DB_USER"); v != "" {
|
||||
cfg.YMTTestDB.User = v
|
||||
}
|
||||
if v := os.Getenv("YMT_TEST_DB_PASSWORD"); v != "" {
|
||||
cfg.YMTTestDB.Password = v
|
||||
}
|
||||
if v := os.Getenv("YMT_TEST_DB_NAME"); v != "" {
|
||||
cfg.YMTTestDB.Name = v
|
||||
}
|
||||
return cfg
|
||||
}
|
||||
|
||||
func readYAML(path string, out *App) bool {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
defer f.Close()
|
||||
b, err := io.ReadAll(f)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
var r root
|
||||
if err := yaml.Unmarshal(b, &r); err == nil {
|
||||
if r.App.Port != "" || r.App.MarketingDB.Host != "" || r.App.YMTDB.Host != "" {
|
||||
*out = r.App
|
||||
return true
|
||||
}
|
||||
}
|
||||
if err := yaml.Unmarshal(b, out); err != nil {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func (d DB) DSN() string {
|
||||
if strings.TrimSpace(d.User) == "" || strings.TrimSpace(d.Host) == "" || strings.TrimSpace(d.Port) == "" || strings.TrimSpace(d.Name) == "" {
|
||||
return ""
|
||||
}
|
||||
return d.User + ":" + d.Password + "@tcp(" + d.Host + ":" + d.Port + ")/" + d.Name + "?parseTime=True&loc=Local&charset=utf8mb4"
|
||||
}
|
||||
|
||||
func LoadEnv() {
|
||||
// optional local env override only
|
||||
paths := []string{
|
||||
".env.local",
|
||||
filepath.Join("server", ".env.local"),
|
||||
}
|
||||
for _, p := range paths {
|
||||
f, err := os.Open(p)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
b, err := io.ReadAll(f)
|
||||
f.Close()
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
lines := strings.Split(string(b), "\n")
|
||||
for _, ln := range lines {
|
||||
s := strings.TrimSpace(ln)
|
||||
if s == "" || strings.HasPrefix(s, "#") {
|
||||
continue
|
||||
}
|
||||
kv := strings.SplitN(s, "=", 2)
|
||||
if len(kv) != 2 {
|
||||
continue
|
||||
}
|
||||
k := strings.TrimSpace(kv[0])
|
||||
v := strings.TrimSpace(kv[1])
|
||||
if k != "" {
|
||||
os.Setenv(k, v)
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
package db
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
|
||||
_ "github.com/go-sql-driver/mysql"
|
||||
)
|
||||
|
||||
func ConnectMySQL(dsn string) (*sql.DB, error) {
|
||||
db, err := sql.Open("mysql", dsn)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := db.Ping(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return db, nil
|
||||
}
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
package db
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"os"
|
||||
"strconv"
|
||||
"time"
|
||||
)
|
||||
|
||||
func ApplyPoolFromEnv(d *sql.DB, prefix string) {
|
||||
if d == nil {
|
||||
return
|
||||
}
|
||||
if v := os.Getenv(prefix + "MAX_OPEN_CONNS"); v != "" {
|
||||
if n, err := strconv.Atoi(v); err == nil && n > 0 {
|
||||
d.SetMaxOpenConns(n)
|
||||
}
|
||||
}
|
||||
if v := os.Getenv(prefix + "MAX_IDLE_CONNS"); v != "" {
|
||||
if n, err := strconv.Atoi(v); err == nil && n >= 0 {
|
||||
d.SetMaxIdleConns(n)
|
||||
}
|
||||
}
|
||||
if v := os.Getenv(prefix + "CONN_MAX_LIFETIME"); v != "" {
|
||||
if dur, err := time.ParseDuration(v); err == nil {
|
||||
d.SetConnMaxLifetime(dur)
|
||||
}
|
||||
}
|
||||
if v := os.Getenv(prefix + "CONN_MAX_IDLE_TIME"); v != "" {
|
||||
if dur, err := time.ParseDuration(v); err == nil {
|
||||
d.SetConnMaxIdleTime(dur)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,76 @@
|
|||
package exporter
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"strings"
|
||||
"server/internal/schema"
|
||||
)
|
||||
|
||||
func EvaluateExplain(db *sql.DB, q string, args []interface{}) (int, []string, error) {
|
||||
rows, score, err := RunExplain(db, q, args)
|
||||
if err != nil { return 0, nil, err }
|
||||
sugg := []string{}
|
||||
for _, r := range rows {
|
||||
// tbl := r.Table.String
|
||||
typ := r.Type.String
|
||||
if typ == "" && r.SelectType.Valid { typ = r.SelectType.String }
|
||||
if typ == "ALL" {
|
||||
sugg = append(sugg, "出现全表扫描(ALL)，请在过滤或连接列上建立索引")
|
||||
}
|
||||
if r.Extra.Valid {
|
||||
e := r.Extra.String
|
||||
if contains(e, "Using temporary") || contains(e, "Using filesort") {
|
||||
sugg = append(sugg, "出现临时表或文件排序，请优化排序列及索引覆盖")
|
||||
}
|
||||
}
|
||||
}
|
||||
return score, sugg, nil
|
||||
}
|
||||
|
||||
func IndexSuggestions(req BuildRequest) []string {
|
||||
sugg := []string{}
|
||||
sch := schema.Get(req.Datasource, req.MainTable)
|
||||
// Filter-based suggestions
|
||||
has := func(k string) bool { _, ok := req.Filters[k]; return ok }
|
||||
add := func(s string){ if s != "" { sugg = append(sugg, s) } }
|
||||
if has("creator_in") && has("create_time_between") {
|
||||
add(fmt.Sprintf("建议在 `%s`(create_time, %s) 建立复合索引以覆盖权限与时间范围", sch.TableName("order"), colName(sch, "creator_in")))
|
||||
} else {
|
||||
if has("creator_in") { add(fmt.Sprintf("建议在 `%s`(%s) 建立索引以覆盖权限过滤", sch.TableName("order"), colName(sch, "creator_in"))) }
|
||||
if has("create_time_between") { add(fmt.Sprintf("建议在 `%s`(create_time) 建立索引以覆盖时间范围", sch.TableName("order"))) }
|
||||
}
|
||||
if has("plan_id_eq") { add(fmt.Sprintf("建议在 `%s`(%s) 建立索引以覆盖活动/计划过滤", sch.TableName("order"), colName(sch, "plan_id_eq"))) }
|
||||
if has("reseller_id_eq") { add(fmt.Sprintf("建议在 `%s`(%s) 建立索引以覆盖分销商过滤", sch.TableName("order"), colName(sch, "reseller_id_eq"))) }
|
||||
if has("product_id_eq") { add(fmt.Sprintf("建议在 `%s`(%s) 建立索引以覆盖商品过滤", sch.TableName("order"), colName(sch, "product_id_eq"))) }
|
||||
if has("out_trade_no_eq") { add(fmt.Sprintf("建议在 `%s`(%s) 建立索引以覆盖支付流水过滤", sch.TableName("order"), colName(sch, "out_trade_no_eq"))) }
|
||||
// Table usage-based join suggestions
|
||||
usedTables := map[string]bool{}
|
||||
for _, tf := range req.Fields {
|
||||
parts := strings.Split(tf, ".")
|
||||
if len(parts)==2 { usedTables[parts[0]] = true }
|
||||
}
|
||||
if req.MainTable == "order_info" {
|
||||
add("建议在 `order_info`(order_no) 建立索引以优化与子表的连接")
|
||||
if usedTables["order_cash"] { add("建议在 `order_cash`(order_no) 建立索引以优化与主表的连接") }
|
||||
if usedTables["order_voucher"] { add("建议在 `order_voucher`(order_no) 建立索引以优化与主表的连接") }
|
||||
if usedTables["order_digit"] { add("建议在 `order_digit`(order_no) 建立索引以优化与主表的连接") }
|
||||
if usedTables["goods_voucher_batch"] { add("建议在 `goods_voucher_batch`(channel_batch_no) 建立索引以优化与订单立减金的连接") }
|
||||
if usedTables["goods_voucher_subject_config"] { add("建议在 `goods_voucher_subject_config`(id) 上确保主键索引以优化连接") }
|
||||
if usedTables["merchant"] { add("建议在 `merchant`(id) 上确保主键索引以优化连接") }
|
||||
if usedTables["activity"] { add("建议在 `activity`(id) 上确保主键索引以优化连接") }
|
||||
}
|
||||
return dedup(sugg)
|
||||
}
|
||||
|
||||
func colName(sch schema.Schema, key string) string {
|
||||
if _, col, ok := sch.FilterColumn(key); ok { return col }
|
||||
return ""
|
||||
}
|
||||
|
||||
func dedup(arr []string) []string {
|
||||
m := map[string]bool{}
|
||||
out := []string{}
|
||||
for _, s := range arr { if !m[s] { m[s]=true; out = append(out, s) } }
|
||||
return out
|
||||
}
|
||||
|
|
@ -0,0 +1,158 @@
|
|||
package exporter
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"log"
|
||||
)
|
||||
|
||||
type ExplainRow struct {
|
||||
ID sql.NullInt64
|
||||
SelectType sql.NullString
|
||||
Table sql.NullString
|
||||
Type sql.NullString
|
||||
PossibleKeys sql.NullString
|
||||
Key sql.NullString
|
||||
KeyLen sql.NullString
|
||||
Ref sql.NullString
|
||||
Rows sql.NullInt64
|
||||
Filtered sql.NullFloat64
|
||||
Extra sql.NullString
|
||||
}
|
||||
|
||||
func RunExplain(db *sql.DB, q string, args []interface{}) ([]ExplainRow, int, error) {
|
||||
log.Printf("sql=EXPLAIN %s args=%v", q, args)
|
||||
rows, err := db.Query("EXPLAIN "+q, args...)
|
||||
if err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
defer rows.Close()
|
||||
res := []ExplainRow{}
|
||||
cols, _ := rows.Columns()
|
||||
for rows.Next() {
|
||||
vals := make([]interface{}, len(cols))
|
||||
dest := make([]interface{}, len(cols))
|
||||
for i := range vals {
|
||||
dest[i] = &vals[i]
|
||||
}
|
||||
if err := rows.Scan(dest...); err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
r := ExplainRow{}
|
||||
if len(cols) >= 10 {
|
||||
toRow(vals, &r)
|
||||
}
|
||||
res = append(res, r)
|
||||
}
|
||||
score := 100
|
||||
for _, r := range res {
|
||||
if r.Type.String == "ALL" {
|
||||
score -= 50
|
||||
}
|
||||
if r.Rows.Int64 > 1000000 {
|
||||
score -= 30
|
||||
}
|
||||
if r.Extra.Valid {
|
||||
if contains(r.Extra.String, "Using temporary") || contains(r.Extra.String, "Using filesort") {
|
||||
score -= 20
|
||||
}
|
||||
}
|
||||
}
|
||||
if score < 0 {
|
||||
score = 0
|
||||
}
|
||||
return res, score, nil
|
||||
}
|
||||
|
||||
func toRow(vals []interface{}, r *ExplainRow) {
|
||||
if s, ok := vals[0].([]byte); ok {
|
||||
r.ID.Int64 = toInt64(string(s))
|
||||
r.ID.Valid = true
|
||||
}
|
||||
if s, ok := vals[1].([]byte); ok {
|
||||
r.SelectType.String = string(s)
|
||||
r.SelectType.Valid = true
|
||||
}
|
||||
if s, ok := vals[2].([]byte); ok {
|
||||
r.Table.String = string(s)
|
||||
r.Table.Valid = true
|
||||
}
|
||||
if s, ok := vals[3].([]byte); ok {
|
||||
r.Type.String = string(s)
|
||||
r.Type.Valid = true
|
||||
}
|
||||
if s, ok := vals[4].([]byte); ok {
|
||||
r.PossibleKeys.String = string(s)
|
||||
r.PossibleKeys.Valid = true
|
||||
}
|
||||
if s, ok := vals[5].([]byte); ok {
|
||||
r.Key.String = string(s)
|
||||
r.Key.Valid = true
|
||||
}
|
||||
if s, ok := vals[6].([]byte); ok {
|
||||
r.KeyLen.String = string(s)
|
||||
r.KeyLen.Valid = true
|
||||
}
|
||||
if s, ok := vals[7].([]byte); ok {
|
||||
r.Ref.String = string(s)
|
||||
r.Ref.Valid = true
|
||||
}
|
||||
if s, ok := vals[8].([]byte); ok {
|
||||
r.Rows.Int64 = toInt64(string(s))
|
||||
r.Rows.Valid = true
|
||||
}
|
||||
if s, ok := vals[9].([]byte); ok {
|
||||
r.Filtered.Float64 = toFloat64(string(s))
|
||||
r.Filtered.Valid = true
|
||||
}
|
||||
if len(vals) > 10 {
|
||||
if s, ok := vals[10].([]byte); ok {
|
||||
r.Extra.String = string(s)
|
||||
r.Extra.Valid = true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func contains(s, sub string) bool {
|
||||
for i := 0; i+len(sub) <= len(s); i++ {
|
||||
if s[i:i+len(sub)] == sub {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func toInt64(s string) int64 {
|
||||
var n int64
|
||||
for i := 0; i < len(s); i++ {
|
||||
c := s[i]
|
||||
if c < '0' || c > '9' {
|
||||
continue
|
||||
}
|
||||
n = n*10 + int64(c-'0')
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
func toFloat64(s string) float64 {
|
||||
var n float64
|
||||
var d float64
|
||||
var seen bool
|
||||
d = 1
|
||||
for i := 0; i < len(s); i++ {
|
||||
c := s[i]
|
||||
if c == '.' {
|
||||
seen = true
|
||||
continue
|
||||
}
|
||||
if c < '0' || c > '9' {
|
||||
continue
|
||||
}
|
||||
if !seen {
|
||||
n = n*10 + float64(c-'0')
|
||||
} else {
|
||||
d *= 10
|
||||
n = n + float64(c-'0')/d
|
||||
}
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
|
@ -0,0 +1,518 @@
|
|||
package exporter
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"server/internal/schema"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
type BuildRequest struct {
|
||||
MainTable string
|
||||
Datasource string
|
||||
Fields []string // table.field
|
||||
Filters map[string]interface{}
|
||||
}
|
||||
|
||||
func BuildSQL(req BuildRequest, whitelist map[string]bool) (string, []interface{}, error) {
|
||||
if req.MainTable != "order" && req.MainTable != "order_info" {
|
||||
return "", nil, errors.New("unsupported main table")
|
||||
}
|
||||
sch := schema.Get(req.Datasource, req.MainTable)
|
||||
if req.Datasource == "marketing" && req.MainTable == "order" {
|
||||
if v, ok := req.Filters["create_time_between"]; ok {
|
||||
switch t := v.(type) {
|
||||
case []interface{}:
|
||||
if len(t) != 2 {
|
||||
return "", nil, errors.New("create_time_between 需要两个时间值")
|
||||
}
|
||||
case []string:
|
||||
if len(t) != 2 {
|
||||
return "", nil, errors.New("create_time_between 需要两个时间值")
|
||||
}
|
||||
default:
|
||||
return "", nil, errors.New("create_time_between 格式错误")
|
||||
}
|
||||
} else {
|
||||
return "", nil, errors.New("缺少时间过滤：必须提供 create_time_between")
|
||||
}
|
||||
}
|
||||
cols := []string{}
|
||||
need := map[string]bool{}
|
||||
for _, tf := range req.Fields {
|
||||
// normalize YMT physical names saved previously to logical names
|
||||
if req.Datasource == "ymt" && strings.HasPrefix(tf, "order_info.") {
|
||||
tf = strings.Replace(tf, "order_info.", "order.", 1)
|
||||
}
|
||||
if whitelist != nil && !whitelist[tf] {
|
||||
continue
|
||||
}
|
||||
parts := strings.Split(tf, ".")
|
||||
if len(parts) != 2 {
|
||||
return "", nil, errors.New("invalid field format")
|
||||
}
|
||||
t, f := parts[0], parts[1]
|
||||
if req.Datasource == "marketing" && t == "order_voucher" && f == "channel_batch_no" {
|
||||
f = "channel_activity_id"
|
||||
}
|
||||
if req.Datasource == "ymt" && t == "order_voucher" && f == "channel_activity_id" {
|
||||
f = "channel_batch_no"
|
||||
}
|
||||
need[t] = true
|
||||
mt := sch.TableName(t)
|
||||
mf, _ := sch.MapField(t, f)
|
||||
if req.Datasource == "marketing" && t == "order" && req.MainTable == "order" {
|
||||
if f == "status" {
|
||||
cols = append(cols, "CASE `order`.status WHEN 0 THEN '待充值' WHEN 1 THEN '充值中' WHEN 2 THEN '已完成' WHEN 3 THEN '充值失败' WHEN 4 THEN '已取消' WHEN 5 THEN '已过期' WHEN 6 THEN '待支付' END AS `order.status`")
|
||||
continue
|
||||
}
|
||||
if f == "type" {
|
||||
cols = append(cols, "CASE `order`.type WHEN 1 THEN '直充卡密' WHEN 2 THEN '立减金' WHEN 3 THEN '红包' ELSE '' END AS `order.type`")
|
||||
continue
|
||||
}
|
||||
if f == "pay_type" {
|
||||
cols = append(cols, "CASE `order`.pay_type WHEN 1 THEN '支付宝' WHEN 5 THEN '微信' ELSE '' END AS `order.pay_type`")
|
||||
continue
|
||||
}
|
||||
if f == "pay_status" {
|
||||
cols = append(cols, "CASE `order`.pay_status WHEN 1 THEN '待支付' WHEN 2 THEN '已支付' WHEN 3 THEN '已退款' ELSE '' END AS `order.pay_status`")
|
||||
continue
|
||||
}
|
||||
if req.Datasource == "marketing" && f == "card_code" {
|
||||
cols = append(cols, "CASE WHEN LENGTH(`order`.card_code) > 10 THEN CONCAT(SUBSTRING(`order`.card_code,1,6),'****',SUBSTRING(`order`.card_code, LENGTH(`order`.card_code)-3, 4)) ELSE `order`.card_code END AS `order.card_code`")
|
||||
continue
|
||||
}
|
||||
}
|
||||
if req.Datasource == "ymt" && t == "order" {
|
||||
if f == "type" {
|
||||
cols = append(cols, "CASE `"+mt+"`.type WHEN 1 THEN '红包订单' WHEN 2 THEN '直充卡密订单' WHEN 3 THEN '立减金订单' ELSE '' END AS `order.type`")
|
||||
continue
|
||||
}
|
||||
if f == "status" {
|
||||
cols = append(cols, "CASE `"+mt+"`.status WHEN 1 THEN '待充值' WHEN 2 THEN '充值中' WHEN 3 THEN '充值成功' WHEN 4 THEN '充值失败' WHEN 5 THEN '已过期' WHEN 6 THEN '已作废' WHEN 7 THEN '已核销' WHEN 8 THEN '核销失败' WHEN 9 THEN '订单重置' WHEN 10 THEN '卡单' ELSE '' END AS `order.status`")
|
||||
continue
|
||||
}
|
||||
if f == "pay_status" {
|
||||
cols = append(cols, "CASE `"+mt+"`.pay_status WHEN 1 THEN '待支付' WHEN 2 THEN '支付中' WHEN 3 THEN '已支付' WHEN 4 THEN '取消支付' WHEN 5 THEN '退款中' WHEN 6 THEN '退款成功' ELSE '' END AS `order.pay_status`")
|
||||
continue
|
||||
}
|
||||
}
|
||||
if req.Datasource == "ymt" && t == "activity" {
|
||||
if f == "settlement_type" {
|
||||
cols = append(cols, "CASE `"+mt+"`.settlement_type WHEN 1 THEN '发放结算' WHEN 2 THEN '打开结算' WHEN 3 THEN '领用结算' WHEN 4 THEN '核销结算' ELSE '' END AS `activity.settlement_type`")
|
||||
continue
|
||||
}
|
||||
}
|
||||
if req.Datasource == "ymt" && t == "order_digit" {
|
||||
if f == "order_type" {
|
||||
cols = append(cols, "CASE `"+mt+"`.order_type WHEN 1 THEN '直充' WHEN 2 THEN '卡密' ELSE '' END AS `order_digit.order_type`")
|
||||
continue
|
||||
}
|
||||
}
|
||||
if t == "order_cash" && f == "receive_status" {
|
||||
cols = append(cols, "CASE `order_cash`.receive_status WHEN 0 THEN '待领取' WHEN 1 THEN '领取中' WHEN 2 THEN '领取成功' WHEN 3 THEN '领取失败' ELSE '' END AS `order_cash.receive_status`")
|
||||
continue
|
||||
}
|
||||
// YMT 的 order_cash 表无 is_confirm 字段，输出占位常量
|
||||
if req.Datasource == "ymt" && t == "order_cash" && f == "is_confirm" {
|
||||
cols = append(cols, "0 AS `order_cash.is_confirm`")
|
||||
continue
|
||||
}
|
||||
if t == "order_cash" && f == "channel" {
|
||||
cols = append(cols, "CASE `order_cash`.channel WHEN 1 THEN '支付宝' WHEN 2 THEN '微信' WHEN 3 THEN '云闪付' ELSE '' END AS `order_cash.channel`")
|
||||
continue
|
||||
}
|
||||
if t == "order_voucher" && f == "channel" {
|
||||
cols = append(cols, "CASE `order_voucher`.channel WHEN 1 THEN '支付宝' WHEN 2 THEN '微信' WHEN 3 THEN '云闪付' ELSE '' END AS `order_voucher.channel`")
|
||||
continue
|
||||
}
|
||||
if req.Datasource == "ymt" && t == "order_voucher" && f == "status" {
|
||||
cols = append(cols, "CASE `order_voucher`.status WHEN 1 THEN '待发放' WHEN 2 THEN '发放中' WHEN 3 THEN '发放失败' WHEN 4 THEN '待核销' WHEN 5 THEN '已核销' WHEN 6 THEN '已过期' WHEN 7 THEN '已退款' ELSE '' END AS `order_voucher.status`")
|
||||
continue
|
||||
}
|
||||
if t == "order_voucher" && f == "status" {
|
||||
cols = append(cols, "CASE `order_voucher`.status WHEN 1 THEN '可用' WHEN 2 THEN '已实扣' WHEN 3 THEN '已过期' WHEN 4 THEN '已退款' WHEN 5 THEN '领取失败' WHEN 6 THEN '发放中' WHEN 7 THEN '部分退款' WHEN 8 THEN '已退回' WHEN 9 THEN '发放失败' ELSE '' END AS `order_voucher.status`")
|
||||
continue
|
||||
}
|
||||
if t == "order_voucher" && f == "receive_mode" {
|
||||
cols = append(cols, "CASE `order_voucher`.receive_mode WHEN 1 THEN '渠道授权用户id' WHEN 2 THEN '手机号或邮箱' ELSE '' END AS `order_voucher.receive_mode`")
|
||||
continue
|
||||
}
|
||||
if t == "order_voucher" && f == "out_biz_no" {
|
||||
cols = append(cols, "'' AS `order_voucher.out_biz_no`")
|
||||
continue
|
||||
}
|
||||
// Fallback for YMT tables that are not joined in schema: voucher, voucher_batch, merchant_key_send
|
||||
if req.Datasource == "ymt" && (t == "voucher" || t == "voucher_batch" || t == "merchant_key_send") {
|
||||
cols = append(cols, "'' AS `"+t+"."+f+"`")
|
||||
continue
|
||||
}
|
||||
cols = append(cols, "`"+mt+"`."+escape(mf)+" AS `"+t+"."+f+"`")
|
||||
}
|
||||
if len(cols) == 0 {
|
||||
return "", nil, errors.New("no fields")
|
||||
}
|
||||
sb := strings.Builder{}
|
||||
baseCols := strings.Join(cols, ",")
|
||||
sb.WriteString("SELECT ")
|
||||
sb.WriteString(baseCols)
|
||||
sb.WriteString(" FROM `" + sch.TableName(req.MainTable) + "`")
|
||||
|
||||
args := []interface{}{}
|
||||
where := []string{}
|
||||
// collect need from filters referencing related tables
|
||||
if _, ok := req.Filters["order_cash_cash_activity_id_eq"]; ok {
|
||||
need["order_cash"] = true
|
||||
}
|
||||
if _, ok := req.Filters["order_voucher_channel_activity_id_eq"]; ok {
|
||||
need["order_voucher"] = true
|
||||
}
|
||||
if _, ok := req.Filters["voucher_batch_channel_activity_id_eq"]; ok {
|
||||
need["voucher_batch"] = true
|
||||
need["voucher"] = true
|
||||
need["order_voucher"] = true
|
||||
}
|
||||
if _, ok := req.Filters["merchant_out_biz_no_eq"]; ok {
|
||||
need["merchant_key_send"] = true
|
||||
}
|
||||
if v, ok := req.Filters["creator_in"]; ok {
|
||||
ids := []interface{}{}
|
||||
switch t := v.(type) {
|
||||
case []interface{}:
|
||||
ids = t
|
||||
case []int:
|
||||
for _, x := range t {
|
||||
ids = append(ids, x)
|
||||
}
|
||||
case []string:
|
||||
for _, x := range t {
|
||||
ids = append(ids, x)
|
||||
}
|
||||
}
|
||||
if len(ids) > 0 {
|
||||
ph := strings.Repeat("?,", len(ids))
|
||||
ph = strings.TrimSuffix(ph, ",")
|
||||
if tbl, col, ok := sch.FilterColumn("creator_in"); ok {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s IN (%s)", sch.TableName(tbl), escape(col), ph))
|
||||
}
|
||||
args = append(args, ids...)
|
||||
}
|
||||
}
|
||||
if v, ok := req.Filters["create_time_between"]; ok {
|
||||
var arr []interface{}
|
||||
b, _ := json.Marshal(v)
|
||||
json.Unmarshal(b, &arr)
|
||||
if len(arr) != 2 {
|
||||
return "", nil, errors.New("create_time_between requires 2 values")
|
||||
}
|
||||
if tbl, col, ok := sch.FilterColumn("create_time_between"); ok {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s BETWEEN ? AND ?", sch.TableName(tbl), escape(col)))
|
||||
}
|
||||
args = append(args, arr[0], arr[1])
|
||||
}
|
||||
if v, ok := req.Filters["type_eq"]; ok {
|
||||
var tv int
|
||||
switch t := v.(type) {
|
||||
case float64:
|
||||
tv = int(t)
|
||||
case int:
|
||||
tv = t
|
||||
case string:
|
||||
// simple digits parsing and label mapping
|
||||
_s := strings.TrimSpace(t)
|
||||
for i := 0; i < len(_s); i++ {
|
||||
c := _s[i]
|
||||
if c < '0' || c > '9' {
|
||||
continue
|
||||
}
|
||||
tv = tv*10 + int(c-'0')
|
||||
}
|
||||
if tv == 0 {
|
||||
if req.Datasource == "ymt" {
|
||||
if _s == "红包订单" {
|
||||
tv = 1
|
||||
}
|
||||
if _s == "直充卡密订单" {
|
||||
tv = 2
|
||||
}
|
||||
if _s == "立减金订单" {
|
||||
tv = 3
|
||||
}
|
||||
} else {
|
||||
if _s == "直充卡密" {
|
||||
tv = 1
|
||||
}
|
||||
if _s == "立减金" {
|
||||
tv = 2
|
||||
}
|
||||
if _s == "红包" {
|
||||
tv = 3
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if tv == 1 || tv == 2 || tv == 3 {
|
||||
if tbl, col, ok := sch.FilterColumn("type_eq"); ok {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s = ?", sch.TableName(tbl), escape(col)))
|
||||
}
|
||||
args = append(args, tv)
|
||||
}
|
||||
}
|
||||
if v, ok := req.Filters["out_trade_no_eq"]; ok {
|
||||
s := toString(v)
|
||||
if s != "" {
|
||||
if tbl, col, ok := sch.FilterColumn("out_trade_no_eq"); ok {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s = ?", sch.TableName(tbl), escape(col)))
|
||||
}
|
||||
args = append(args, s)
|
||||
}
|
||||
}
|
||||
if v, ok := req.Filters["account_eq"]; ok {
|
||||
s := toString(v)
|
||||
if s != "" {
|
||||
if tbl, col, ok := sch.FilterColumn("account_eq"); ok {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s = ?", sch.TableName(tbl), escape(col)))
|
||||
}
|
||||
args = append(args, s)
|
||||
}
|
||||
}
|
||||
if v, ok := req.Filters["plan_id_eq"]; ok {
|
||||
s := toString(v)
|
||||
if s != "" {
|
||||
if tbl, col, ok := sch.FilterColumn("plan_id_eq"); ok {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s = ?", sch.TableName(tbl), escape(col)))
|
||||
}
|
||||
args = append(args, s)
|
||||
}
|
||||
}
|
||||
if v, ok := req.Filters["key_batch_id_eq"]; ok {
|
||||
s := toString(v)
|
||||
if s != "" {
|
||||
if tbl, col, ok := sch.FilterColumn("key_batch_id_eq"); ok {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s = ?", sch.TableName(tbl), escape(col)))
|
||||
}
|
||||
args = append(args, s)
|
||||
}
|
||||
}
|
||||
if v, ok := req.Filters["product_id_eq"]; ok {
|
||||
s := toString(v)
|
||||
if s != "" {
|
||||
if tbl, col, ok := sch.FilterColumn("product_id_eq"); ok {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s = ?", sch.TableName(tbl), escape(col)))
|
||||
}
|
||||
args = append(args, s)
|
||||
}
|
||||
}
|
||||
if v, ok := req.Filters["reseller_id_eq"]; ok {
|
||||
s := toString(v)
|
||||
if s != "" {
|
||||
if tbl, col, ok := sch.FilterColumn("reseller_id_eq"); ok {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s = ?", sch.TableName(tbl), escape(col)))
|
||||
}
|
||||
args = append(args, s)
|
||||
}
|
||||
}
|
||||
if v, ok := req.Filters["code_batch_id_eq"]; ok {
|
||||
s := toString(v)
|
||||
if s != "" {
|
||||
if tbl, col, ok := sch.FilterColumn("code_batch_id_eq"); ok {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s = ?", sch.TableName(tbl), escape(col)))
|
||||
}
|
||||
args = append(args, s)
|
||||
}
|
||||
}
|
||||
if v, ok := req.Filters["order_cash_cash_activity_id_eq"]; ok {
|
||||
s := toString(v)
|
||||
if s != "" {
|
||||
if tbl, col, ok := sch.FilterColumn("order_cash_cash_activity_id_eq"); ok {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s = ?", sch.TableName(tbl), escape(col)))
|
||||
}
|
||||
args = append(args, s)
|
||||
}
|
||||
}
|
||||
if v, ok := req.Filters["order_voucher_channel_activity_id_eq"]; ok {
|
||||
s := toString(v)
|
||||
if s != "" {
|
||||
if tbl, col, ok := sch.FilterColumn("order_voucher_channel_activity_id_eq"); ok {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s = ?", sch.TableName(tbl), escape(col)))
|
||||
}
|
||||
args = append(args, s)
|
||||
}
|
||||
}
|
||||
if v, ok := req.Filters["voucher_batch_channel_activity_id_eq"]; ok {
|
||||
s := toString(v)
|
||||
if s != "" {
|
||||
if tbl, col, ok := sch.FilterColumn("voucher_batch_channel_activity_id_eq"); ok {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s = ?", sch.TableName(tbl), escape(col)))
|
||||
args = append(args, s)
|
||||
}
|
||||
}
|
||||
}
|
||||
if v, ok := req.Filters["merchant_out_biz_no_eq"]; ok {
|
||||
s := toString(v)
|
||||
if s != "" {
|
||||
if tbl, col, ok := sch.FilterColumn("merchant_out_biz_no_eq"); ok {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s = ?", sch.TableName(tbl), escape(col)))
|
||||
}
|
||||
args = append(args, s)
|
||||
}
|
||||
}
|
||||
// append necessary joins after collecting filter-driven needs
|
||||
joins := sch.BuildJoins(need, req.MainTable)
|
||||
for _, j := range joins {
|
||||
sb.WriteString(j)
|
||||
}
|
||||
if len(where) > 0 {
|
||||
sb.WriteString(" WHERE ")
|
||||
sb.WriteString(strings.Join(where, " AND "))
|
||||
}
|
||||
needDedupe := req.Datasource == "marketing" && req.MainTable == "order" && (need["order_voucher"] || need["voucher"] || need["voucher_batch"] || need["key_batch"] || need["code_batch"])
|
||||
if needDedupe {
|
||||
// Extract alias names in order
|
||||
aliases := make([]string, 0, len(cols))
|
||||
sources := make([]string, 0, len(cols))
|
||||
for _, c := range cols {
|
||||
pos := strings.LastIndex(c, " AS `")
|
||||
if pos < 0 {
|
||||
continue
|
||||
}
|
||||
alias := c[pos+5:]
|
||||
if len(alias) == 0 {
|
||||
continue
|
||||
}
|
||||
if alias[len(alias)-1] == '`' {
|
||||
alias = alias[:len(alias)-1]
|
||||
}
|
||||
aliases = append(aliases, alias)
|
||||
parts := strings.Split(alias, ".")
|
||||
src := ""
|
||||
if len(parts) >= 1 {
|
||||
src = parts[0]
|
||||
}
|
||||
sources = append(sources, src)
|
||||
}
|
||||
mt := sch.TableName(req.MainTable)
|
||||
pkCol, _ := sch.MapField(req.MainTable, "order_number")
|
||||
var out strings.Builder
|
||||
out.WriteString("SELECT ")
|
||||
for i := range aliases {
|
||||
if i > 0 {
|
||||
out.WriteString(",")
|
||||
}
|
||||
alias := aliases[i]
|
||||
src := sources[i]
|
||||
if src == "order" {
|
||||
out.WriteString("sub.`" + alias + "`")
|
||||
} else {
|
||||
out.WriteString("MIN(sub.`" + alias + "`) AS `" + alias + "`")
|
||||
}
|
||||
}
|
||||
out.WriteString(" FROM (")
|
||||
out.WriteString(sb.String())
|
||||
out.WriteString(") AS sub GROUP BY sub.`" + mt + "." + pkCol + "`")
|
||||
return out.String(), args, nil
|
||||
}
|
||||
return sb.String(), args, nil
|
||||
}
|
||||
|
||||
func escape(s string) string {
|
||||
if s == "key" {
|
||||
return "`key`"
|
||||
}
|
||||
if s == "index" {
|
||||
return "`index`"
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func toString(v interface{}) string {
|
||||
switch t := v.(type) {
|
||||
case []byte:
|
||||
return string(t)
|
||||
case string:
|
||||
return t
|
||||
case int64:
|
||||
return strconv.FormatInt(t, 10)
|
||||
case int:
|
||||
return strconv.Itoa(t)
|
||||
case float64:
|
||||
return strconv.FormatFloat(t, 'f', -1, 64)
|
||||
case time.Time:
|
||||
return t.Format("2006-01-02 15:04:05")
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
// BuildCountSQL: minimal COUNT for filters-only joins, counting distinct main PK to avoid 1:N duplication
|
||||
func BuildCountSQL(req BuildRequest, whitelist map[string]bool) (string, []interface{}, error) {
|
||||
if req.MainTable != "order" && req.MainTable != "order_info" {
|
||||
return "", nil, errors.New("unsupported main table")
|
||||
}
|
||||
sch := schema.Get(req.Datasource, req.MainTable)
|
||||
mt := sch.TableName(req.MainTable)
|
||||
pkCol, _ := sch.MapField(req.MainTable, "order_number")
|
||||
args := []interface{}{}
|
||||
where := []string{}
|
||||
need := map[string]bool{}
|
||||
// mark joins only needed by filters
|
||||
for k, _ := range req.Filters {
|
||||
if tbl, _, ok := sch.FilterColumn(k); ok {
|
||||
need[tbl] = true
|
||||
}
|
||||
}
|
||||
// build WHERE from filters
|
||||
for k, v := range req.Filters {
|
||||
if tbl, col, ok := sch.FilterColumn(k); ok {
|
||||
switch k {
|
||||
case "creator_in":
|
||||
ids := []interface{}{}
|
||||
switch t := v.(type) {
|
||||
case []interface{}:
|
||||
ids = t
|
||||
case []int:
|
||||
for _, x := range t {
|
||||
ids = append(ids, x)
|
||||
}
|
||||
case []string:
|
||||
for _, x := range t {
|
||||
ids = append(ids, x)
|
||||
}
|
||||
}
|
||||
if len(ids) > 0 {
|
||||
ph := strings.Repeat("?,", len(ids))
|
||||
ph = strings.TrimSuffix(ph, ",")
|
||||
where = append(where, fmt.Sprintf("`%s`.%s IN (%s)", sch.TableName(tbl), escape(col), ph))
|
||||
args = append(args, ids...)
|
||||
}
|
||||
case "create_time_between":
|
||||
var arr []interface{}
|
||||
b, _ := json.Marshal(v)
|
||||
_ = json.Unmarshal(b, &arr)
|
||||
if len(arr) == 2 {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s BETWEEN ? AND ?", sch.TableName(tbl), escape(col)))
|
||||
args = append(args, arr[0], arr[1])
|
||||
}
|
||||
default:
|
||||
s := toString(v)
|
||||
if s != "" {
|
||||
where = append(where, fmt.Sprintf("`%s`.%s = ?", sch.TableName(tbl), escape(col)))
|
||||
args = append(args, s)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
sb := strings.Builder{}
|
||||
sb.WriteString("SELECT COUNT(DISTINCT `" + mt + "`." + pkCol + ") FROM `" + mt + "`")
|
||||
for _, j := range sch.BuildJoins(need, req.MainTable) {
|
||||
sb.WriteString(j)
|
||||
}
|
||||
if len(where) > 0 {
|
||||
sb.WriteString(" WHERE ")
|
||||
sb.WriteString(strings.Join(where, " AND "))
|
||||
}
|
||||
return sb.String(), args, nil
|
||||
}
|
||||
|
|
@ -0,0 +1,427 @@
|
|||
package exporter
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"log"
|
||||
"server/internal/logging"
|
||||
"server/internal/schema"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
type CursorSQL struct{ ds, main, mt, tsCol, pkCol string }
|
||||
|
||||
func NewCursorSQL(ds, main string) *CursorSQL {
|
||||
sch := schema.Get(ds, main)
|
||||
mt := sch.TableName(main)
|
||||
ts, _ := sch.MapField(main, "create_time")
|
||||
pk, _ := sch.MapField(main, "order_number")
|
||||
return &CursorSQL{ds: ds, main: main, mt: mt, tsCol: ts, pkCol: pk}
|
||||
}
|
||||
|
||||
func (c *CursorSQL) InjectSelect(base string) string {
|
||||
idx := strings.Index(base, " FROM ")
|
||||
if idx <= 0 {
|
||||
return base
|
||||
}
|
||||
u := strings.ToUpper(base)
|
||||
prefix := "SELECT "
|
||||
if strings.HasPrefix(u, "SELECT DISTINCT ") {
|
||||
prefix = "SELECT DISTINCT "
|
||||
} else if strings.HasPrefix(u, "SELECT SQL_NO_CACHE ") {
|
||||
prefix = "SELECT SQL_NO_CACHE "
|
||||
}
|
||||
return prefix + "`" + c.mt + "`." + c.tsCol + " AS __ts, `" + c.mt + "`." + c.pkCol + " AS __pk, " + base[len(prefix):]
|
||||
}
|
||||
|
||||
func (c *CursorSQL) AddOrder(base string) string {
|
||||
return base + " ORDER BY `" + c.mt + "`." + c.tsCol + ", `" + c.mt + "`." + c.pkCol
|
||||
}
|
||||
|
||||
func (c *CursorSQL) AddCursor(base string) string {
|
||||
u := strings.ToUpper(base)
|
||||
cond := " AND ((`" + c.mt + "`." + c.tsCol + ") > ? OR ((`" + c.mt + "`." + c.tsCol + ") = ? AND (`" + c.mt + "`." + c.pkCol + ") > ?))"
|
||||
if strings.Contains(u, " WHERE ") {
|
||||
return base + cond
|
||||
}
|
||||
return base + strings.TrimPrefix(cond, " AND ")
|
||||
}
|
||||
|
||||
func CountRows(db *sql.DB, base string, args []interface{}) int64 {
|
||||
u := strings.ToUpper(base)
|
||||
idx := strings.Index(u, " FROM ")
|
||||
cut := len(base)
|
||||
if idx > 0 {
|
||||
for _, tok := range []string{" ORDER BY ", " LIMIT ", " OFFSET "} {
|
||||
if p := strings.Index(u[idx:], tok); p >= 0 {
|
||||
cp := idx + p
|
||||
if cp < cut {
|
||||
cut = cp
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
minimal := base
|
||||
if idx > 0 {
|
||||
seg := base[idx:cut]
|
||||
minimal = "SELECT 1" + seg
|
||||
}
|
||||
q := "SELECT COUNT(1) FROM (" + minimal + ") AS sub"
|
||||
row := db.QueryRow(q, args...)
|
||||
var c int64
|
||||
if err := row.Scan(&c); err != nil {
|
||||
logging.JSON("ERROR", map[string]interface{}{"event": "count_error", "error": err.Error(), "sql": q, "args": args})
|
||||
log.Printf("count_error sql=%s args=%v err=%v", q, args, err)
|
||||
return 0
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
func CountRowsFast(db *sql.DB, ds, main string, filters map[string]interface{}) int64 {
|
||||
sch := schema.Get(ds, main)
|
||||
mt := sch.TableName(main)
|
||||
q := "SELECT COUNT(1) FROM `" + mt + "` WHERE 1=1"
|
||||
args := []interface{}{}
|
||||
addIn := func(col string, v interface{}) {
|
||||
switch t := v.(type) {
|
||||
case []interface{}:
|
||||
if len(t) == 0 {
|
||||
return
|
||||
}
|
||||
ph := make([]string, len(t))
|
||||
for i := range t {
|
||||
ph[i] = "?"
|
||||
args = append(args, t[i])
|
||||
}
|
||||
q += " AND `" + col + "` IN (" + strings.Join(ph, ",") + ")"
|
||||
case []string:
|
||||
if len(t) == 0 {
|
||||
return
|
||||
}
|
||||
ph := make([]string, len(t))
|
||||
for i := range t {
|
||||
ph[i] = "?"
|
||||
args = append(args, t[i])
|
||||
}
|
||||
q += " AND `" + col + "` IN (" + strings.Join(ph, ",") + ")"
|
||||
case []int:
|
||||
if len(t) == 0 {
|
||||
return
|
||||
}
|
||||
ph := make([]string, len(t))
|
||||
for i := range t {
|
||||
ph[i] = "?"
|
||||
args = append(args, t[i])
|
||||
}
|
||||
q += " AND `" + col + "` IN (" + strings.Join(ph, ",") + ")"
|
||||
case []int64:
|
||||
if len(t) == 0 {
|
||||
return
|
||||
}
|
||||
ph := make([]string, len(t))
|
||||
for i := range t {
|
||||
ph[i] = "?"
|
||||
args = append(args, t[i])
|
||||
}
|
||||
q += " AND `" + col + "` IN (" + strings.Join(ph, ",") + ")"
|
||||
}
|
||||
}
|
||||
for k, v := range filters {
|
||||
tbl, col, ok := sch.FilterColumn(k)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if tbl != "order" {
|
||||
continue
|
||||
}
|
||||
switch k {
|
||||
case "creator_in":
|
||||
addIn(col, v)
|
||||
case "create_time_between":
|
||||
switch t := v.(type) {
|
||||
case []interface{}:
|
||||
if len(t) == 2 {
|
||||
q += " AND `" + col + "` BETWEEN ? AND ?"
|
||||
args = append(args, t[0], t[1])
|
||||
}
|
||||
case []string:
|
||||
if len(t) == 2 {
|
||||
q += " AND `" + col + "` BETWEEN ? AND ?"
|
||||
args = append(args, t[0], t[1])
|
||||
}
|
||||
}
|
||||
default:
|
||||
q += " AND `" + col + "` = ?"
|
||||
args = append(args, v)
|
||||
}
|
||||
}
|
||||
row := db.QueryRow(q, args...)
|
||||
var c int64
|
||||
if err := row.Scan(&c); err != nil {
|
||||
logging.JSON("ERROR", map[string]interface{}{"event": "count_fast_error", "error": err.Error(), "sql": q, "args": args})
|
||||
log.Printf("count_fast_error sql=%s args=%v err=%v", q, args, err)
|
||||
return 0
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
func CountRowsFastChunked(db *sql.DB, ds, main string, filters map[string]interface{}) int64 {
|
||||
start := ""
|
||||
end := ""
|
||||
if v, ok := filters["create_time_between"]; ok {
|
||||
switch t := v.(type) {
|
||||
case []interface{}:
|
||||
if len(t) == 2 {
|
||||
start = toString(t[0])
|
||||
end = toString(t[1])
|
||||
}
|
||||
case []string:
|
||||
if len(t) == 2 {
|
||||
start = t[0]
|
||||
end = t[1]
|
||||
}
|
||||
}
|
||||
}
|
||||
if start == "" || end == "" {
|
||||
return CountRowsFast(db, ds, main, filters)
|
||||
}
|
||||
ranges := splitDays(start, end, 15)
|
||||
var total int64
|
||||
for _, rg := range ranges {
|
||||
fl := map[string]interface{}{}
|
||||
for k, v := range filters {
|
||||
fl[k] = v
|
||||
}
|
||||
fl["create_time_between"] = []string{rg[0], rg[1]}
|
||||
total += CountRowsFast(db, ds, main, fl)
|
||||
}
|
||||
return total
|
||||
}
|
||||
|
||||
func splitDays(startStr, endStr string, stepDays int) [][2]string {
|
||||
layout := "2006-01-02 15:04:05"
|
||||
s := strings.TrimSpace(startStr)
|
||||
e := strings.TrimSpace(endStr)
|
||||
st, err1 := time.Parse(layout, s)
|
||||
en, err2 := time.Parse(layout, e)
|
||||
if err1 != nil || err2 != nil || !en.After(st) || stepDays <= 0 {
|
||||
return [][2]string{{s, e}}
|
||||
}
|
||||
var out [][2]string
|
||||
cur := st
|
||||
step := time.Duration(stepDays) * 24 * time.Hour
|
||||
for cur.Before(en) {
|
||||
nxt := cur.Add(step)
|
||||
if nxt.After(en) {
|
||||
nxt = en
|
||||
}
|
||||
out = append(out, [2]string{cur.Format(layout), nxt.Format(layout)})
|
||||
cur = nxt
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
type RowTransform func([]string) []string
|
||||
type RollCallback func(path string, size int64, partRows int64) error
|
||||
type ProgressCallback func(totalRows int64) error
|
||||
|
||||
func StreamWithCursor(db *sql.DB, base string, args []interface{}, cur *CursorSQL, batch int, cols []string, newWriter func() (RowWriter, error), transform RowTransform, maxRowsPerFile int64, onRoll RollCallback, onProgress ProgressCallback) (int64, []string, error) {
|
||||
w, err := newWriter()
|
||||
if err != nil {
|
||||
return 0, nil, err
|
||||
}
|
||||
_ = w.WriteHeader(cols)
|
||||
if onProgress != nil {
|
||||
_ = onProgress(0)
|
||||
}
|
||||
out := make([]interface{}, len(cols)+2)
|
||||
dest := make([]interface{}, len(cols)+2)
|
||||
for i := range out {
|
||||
dest[i] = &out[i]
|
||||
}
|
||||
var total int64
|
||||
var part int64
|
||||
var tick int64
|
||||
files := []string{}
|
||||
lastTs := ""
|
||||
lastPk := ""
|
||||
for {
|
||||
q2 := cur.InjectSelect(base)
|
||||
if lastTs != "" || lastPk != "" {
|
||||
q2 = cur.AddCursor(q2)
|
||||
}
|
||||
q2 = cur.AddOrder(q2) + " LIMIT ?"
|
||||
args2 := append([]interface{}{}, args...)
|
||||
if lastTs != "" || lastPk != "" {
|
||||
args2 = append(args2, lastTs, lastTs, lastPk)
|
||||
}
|
||||
args2 = append(args2, batch)
|
||||
rows, e := db.Query(q2, args2...)
|
||||
if e != nil {
|
||||
logging.JSON("ERROR", map[string]interface{}{"event": "cursor_query_error", "sql": q2, "args": args2, "error": e.Error()})
|
||||
log.Printf("cursor_query_error sql=%s args=%v err=%v", q2, args2, e)
|
||||
// fallback to LIMIT/OFFSET pagination when cursor query fails
|
||||
_ = rows
|
||||
_, _, _ = w.Close()
|
||||
return pagedOffset(db, base, args, batch, cols, newWriter, transform, maxRowsPerFile, onRoll, onProgress)
|
||||
}
|
||||
fetched := false
|
||||
for rows.Next() {
|
||||
fetched = true
|
||||
if e := rows.Scan(dest...); e != nil {
|
||||
rows.Close()
|
||||
// fallback to LIMIT/OFFSET when scan fails (likely column mismatch)
|
||||
logging.JSON("ERROR", map[string]interface{}{"event": "cursor_scan_error", "error": e.Error()})
|
||||
log.Printf("cursor_scan_error err=%v", e)
|
||||
_, _, _ = w.Close()
|
||||
return pagedOffset(db, base, args, batch, cols, newWriter, transform, maxRowsPerFile, onRoll, onProgress)
|
||||
}
|
||||
vals := make([]string, len(cols))
|
||||
for i := 0; i < len(cols); i++ {
|
||||
// skip the injected cursor columns (__ts, __pk) at positions 0 and 1
|
||||
idx := i + 2
|
||||
if b, ok := out[idx].([]byte); ok {
|
||||
vals[i] = string(b)
|
||||
} else if out[idx] == nil {
|
||||
vals[i] = ""
|
||||
} else {
|
||||
vals[i] = toString(out[idx])
|
||||
}
|
||||
}
|
||||
if transform != nil {
|
||||
vals = transform(vals)
|
||||
}
|
||||
_ = w.WriteRow(vals)
|
||||
total++
|
||||
part++
|
||||
tick++
|
||||
// update cursor state from injected columns
|
||||
lastTs = toString(out[0])
|
||||
lastPk = toString(out[1])
|
||||
if onProgress != nil && (tick == 1 || tick%200 == 0) {
|
||||
_ = onProgress(total)
|
||||
logging.JSON("INFO", map[string]interface{}{"event": "progress_tick", "total_rows": total})
|
||||
}
|
||||
if part >= maxRowsPerFile {
|
||||
p, sz, _ := w.Close()
|
||||
files = append(files, p)
|
||||
if onRoll != nil {
|
||||
_ = onRoll(p, sz, part)
|
||||
}
|
||||
w, e = newWriter()
|
||||
if e != nil {
|
||||
rows.Close()
|
||||
return total, files, e
|
||||
}
|
||||
_ = w.WriteHeader(cols)
|
||||
part = 0
|
||||
}
|
||||
}
|
||||
rows.Close()
|
||||
if !fetched {
|
||||
break
|
||||
}
|
||||
}
|
||||
p, sz, _ := w.Close()
|
||||
if part > 0 || len(files) == 0 {
|
||||
files = append(files, p)
|
||||
if onRoll != nil {
|
||||
_ = onRoll(p, sz, part)
|
||||
}
|
||||
}
|
||||
if onProgress != nil {
|
||||
_ = onProgress(total)
|
||||
}
|
||||
return total, files, nil
|
||||
}
|
||||
|
||||
// pagedOffset provides a robust fallback using LIMIT/OFFSET without cursor columns
|
||||
func pagedOffset(db *sql.DB, base string, args []interface{}, batch int, cols []string, newWriter func() (RowWriter, error), transform RowTransform, maxRowsPerFile int64, onRoll RollCallback, onProgress ProgressCallback) (int64, []string, error) {
|
||||
w, err := newWriter()
|
||||
if err != nil {
|
||||
return 0, nil, err
|
||||
}
|
||||
_ = w.WriteHeader(cols)
|
||||
if onProgress != nil {
|
||||
_ = onProgress(0)
|
||||
}
|
||||
files := []string{}
|
||||
var total int64
|
||||
var part int64
|
||||
var tick int64
|
||||
for off := 0; ; off += batch {
|
||||
q := "SELECT * FROM (" + base + ") AS sub LIMIT ? OFFSET ?"
|
||||
args2 := append(append([]interface{}{}, args...), batch, off)
|
||||
rows, e := db.Query(q, args2...)
|
||||
if e != nil {
|
||||
logging.JSON("ERROR", map[string]interface{}{"event": "offset_query_error", "sql": q, "args": args2, "error": e.Error()})
|
||||
log.Printf("offset_query_error sql=%s args=%v err=%v", q, args2, e)
|
||||
return total, files, e
|
||||
}
|
||||
fetched := false
|
||||
out := make([]interface{}, len(cols))
|
||||
dest := make([]interface{}, len(cols))
|
||||
for i := range out {
|
||||
dest[i] = &out[i]
|
||||
}
|
||||
for rows.Next() {
|
||||
fetched = true
|
||||
if e := rows.Scan(dest...); e != nil {
|
||||
rows.Close()
|
||||
logging.JSON("ERROR", map[string]interface{}{"event": "offset_scan_error", "error": e.Error()})
|
||||
log.Printf("offset_scan_error err=%v", e)
|
||||
return total, files, e
|
||||
}
|
||||
vals := make([]string, len(cols))
|
||||
for i := 0; i < len(cols); i++ {
|
||||
if b, ok := out[i].([]byte); ok {
|
||||
vals[i] = string(b)
|
||||
} else if out[i] == nil {
|
||||
vals[i] = ""
|
||||
} else {
|
||||
vals[i] = toString(out[i])
|
||||
}
|
||||
}
|
||||
if transform != nil {
|
||||
vals = transform(vals)
|
||||
}
|
||||
_ = w.WriteRow(vals)
|
||||
total++
|
||||
part++
|
||||
tick++
|
||||
if onProgress != nil && (tick == 1 || tick%200 == 0) {
|
||||
_ = onProgress(total)
|
||||
logging.JSON("INFO", map[string]interface{}{"event": "progress_tick", "total_rows": total})
|
||||
}
|
||||
if part >= maxRowsPerFile {
|
||||
p, sz, _ := w.Close()
|
||||
files = append(files, p)
|
||||
if onRoll != nil {
|
||||
_ = onRoll(p, sz, part)
|
||||
}
|
||||
w, e = newWriter()
|
||||
if e != nil {
|
||||
rows.Close()
|
||||
return total, files, e
|
||||
}
|
||||
_ = w.WriteHeader(cols)
|
||||
part = 0
|
||||
}
|
||||
}
|
||||
rows.Close()
|
||||
if !fetched {
|
||||
break
|
||||
}
|
||||
}
|
||||
p, sz, _ := w.Close()
|
||||
if part > 0 || len(files) == 0 {
|
||||
files = append(files, p)
|
||||
if onRoll != nil {
|
||||
_ = onRoll(p, sz, part)
|
||||
}
|
||||
}
|
||||
if onProgress != nil {
|
||||
_ = onProgress(total)
|
||||
}
|
||||
return total, files, nil
|
||||
}
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
package exporter
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ZipFiles 将分片文件打包为zip并返回路径与大小，同时清理源xlsx分片文件
|
||||
func ZipFiles(jobID uint64, files []string) (string, int64) {
|
||||
baseDir := "storage/export"
|
||||
_ = os.MkdirAll(baseDir, 0755)
|
||||
zipPath := filepath.Join(baseDir, fmt.Sprintf("job_%d_%d.zip", jobID, time.Now().Unix()))
|
||||
zf, err := os.Create(zipPath)
|
||||
if err != nil {
|
||||
return zipPath, 0
|
||||
}
|
||||
defer zf.Close()
|
||||
zw := zip.NewWriter(zf)
|
||||
for _, p := range files {
|
||||
f, err := os.Open(p)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
w, err := zw.Create(filepath.Base(p))
|
||||
if err != nil {
|
||||
f.Close()
|
||||
continue
|
||||
}
|
||||
_, _ = io.Copy(w, f)
|
||||
f.Close()
|
||||
}
|
||||
_ = zw.Close()
|
||||
st, err := os.Stat(zipPath)
|
||||
if err != nil {
|
||||
return zipPath, 0
|
||||
}
|
||||
// 清理xlsx分片文件以节省空间
|
||||
for _, fp := range files {
|
||||
if strings.HasSuffix(strings.ToLower(fp), ".xlsx") {
|
||||
_ = os.Remove(fp)
|
||||
}
|
||||
}
|
||||
return zipPath, st.Size()
|
||||
}
|
||||
|
|
@ -0,0 +1,157 @@
|
|||
package exporter
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/csv"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"time"
|
||||
|
||||
"github.com/xuri/excelize/v2"
|
||||
)
|
||||
|
||||
type RowWriter interface {
|
||||
WriteHeader(cols []string) error
|
||||
WriteRow(vals []string) error
|
||||
Close() (string, int64, error)
|
||||
}
|
||||
|
||||
type CSVWriter struct {
|
||||
f *os.File
|
||||
buf *bufio.Writer
|
||||
w *csv.Writer
|
||||
count int64
|
||||
}
|
||||
|
||||
func NewCSVWriter(dir, name string) (*CSVWriter, error) {
|
||||
os.MkdirAll(dir, 0755)
|
||||
p := filepath.Join(dir, name+"_"+time.Now().Format("20060102150405")+".csv")
|
||||
f, err := os.Create(p)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
buf := bufio.NewWriterSize(f, 4<<20)
|
||||
return &CSVWriter{f: f, buf: buf, w: csv.NewWriter(buf)}, nil
|
||||
}
|
||||
|
||||
func (c *CSVWriter) WriteHeader(cols []string) error {
|
||||
if err := c.w.Write(cols); err != nil {
|
||||
return err
|
||||
}
|
||||
c.count++
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *CSVWriter) WriteRow(vals []string) error {
|
||||
if err := c.w.Write(vals); err != nil {
|
||||
return err
|
||||
}
|
||||
c.count++
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *CSVWriter) Close() (string, int64, error) {
|
||||
c.w.Flush()
|
||||
if c.buf != nil {
|
||||
_ = c.buf.Flush()
|
||||
}
|
||||
p := c.f.Name()
|
||||
info, _ := c.f.Stat()
|
||||
c.f.Close()
|
||||
return p, info.Size(), nil
|
||||
}
|
||||
|
||||
type XLSXWriter struct {
|
||||
f *excelize.File
|
||||
sheet string
|
||||
row int
|
||||
path string
|
||||
sw *excelize.StreamWriter
|
||||
}
|
||||
|
||||
func NewXLSXWriter(dir, name, sheet string) (*XLSXWriter, error) {
|
||||
os.MkdirAll(dir, 0755)
|
||||
p := filepath.Join(dir, name+"_"+time.Now().Format("20060102150405")+".xlsx")
|
||||
f := excelize.NewFile()
|
||||
idx, err := f.GetSheetIndex(sheet)
|
||||
if err != nil || idx < 0 {
|
||||
idx, _ = f.NewSheet(sheet)
|
||||
f.SetActiveSheet(idx)
|
||||
if sheet != "Sheet1" {
|
||||
_ = f.DeleteSheet("Sheet1")
|
||||
}
|
||||
} else {
|
||||
f.SetActiveSheet(idx)
|
||||
}
|
||||
sw, e := f.NewStreamWriter(sheet)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
return &XLSXWriter{f: f, sheet: sheet, row: 1, path: p, sw: sw}, nil
|
||||
}
|
||||
|
||||
func (x *XLSXWriter) WriteHeader(cols []string) error {
|
||||
vals := make([]interface{}, len(cols))
|
||||
for i := range cols {
|
||||
vals[i] = cols[i]
|
||||
}
|
||||
axis := "A" + itoa(1)
|
||||
if err := x.sw.SetRow(axis, vals); err != nil {
|
||||
return err
|
||||
}
|
||||
x.row = 2
|
||||
return nil
|
||||
}
|
||||
|
||||
func (x *XLSXWriter) WriteRow(vals []string) error {
|
||||
rowVals := make([]interface{}, len(vals))
|
||||
for i := range vals {
|
||||
rowVals[i] = vals[i]
|
||||
}
|
||||
axis := "A" + itoa(x.row)
|
||||
if err := x.sw.SetRow(axis, rowVals); err != nil {
|
||||
return err
|
||||
}
|
||||
x.row++
|
||||
return nil
|
||||
}
|
||||
|
||||
func (x *XLSXWriter) Close() (string, int64, error) {
|
||||
if x.sw != nil {
|
||||
_ = x.sw.Flush()
|
||||
}
|
||||
if err := x.f.SaveAs(x.path); err != nil {
|
||||
return "", 0, err
|
||||
}
|
||||
info, err := os.Stat(x.path)
|
||||
if err != nil {
|
||||
return x.path, 0, nil
|
||||
}
|
||||
return x.path, info.Size(), nil
|
||||
}
|
||||
|
||||
func col(n int) string {
|
||||
s := ""
|
||||
for n > 0 {
|
||||
n--
|
||||
s = string(rune('A'+(n%26))) + s
|
||||
n /= 26
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func itoa(n int) string {
|
||||
if n == 0 {
|
||||
return "0"
|
||||
}
|
||||
b := make([]byte, 0, 10)
|
||||
m := n
|
||||
for m > 0 {
|
||||
b = append(b, byte('0'+(m%10)))
|
||||
m /= 10
|
||||
}
|
||||
for i, j := 0, len(b)-1; i < j; i, j = i+1, j-1 {
|
||||
b[i], b[j] = b[j], b[i]
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
package logging
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"time"
|
||||
)
|
||||
|
||||
func Init(dir string) error {
|
||||
if dir == "" {
|
||||
dir = "log"
|
||||
}
|
||||
if err := os.MkdirAll(dir, 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
name := fmt.Sprintf("server-%s.log", time.Now().Format("20060102"))
|
||||
p := filepath.Join(dir, name)
|
||||
f, err := os.OpenFile(p, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
mw := io.MultiWriter(os.Stdout, f)
|
||||
log.SetOutput(mw)
|
||||
log.SetFlags(0)
|
||||
return nil
|
||||
}
|
||||
|
||||
func JSON(level string, fields map[string]interface{}) {
|
||||
m := map[string]interface{}{"level": level, "ts": time.Now().Format(time.RFC3339)}
|
||||
for k, v := range fields {
|
||||
m[k] = v
|
||||
}
|
||||
b, _ := json.Marshal(m)
|
||||
log.Println(string(b))
|
||||
}
|
||||
|
|
@ -0,0 +1,92 @@
|
|||
package migrate
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
)
|
||||
|
||||
func Apply(db *sql.DB) error {
|
||||
stmts := []string{
|
||||
"CREATE TABLE IF NOT EXISTS export_templates (id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY, name VARCHAR(255) NOT NULL, datasource VARCHAR(32) NOT NULL, main_table VARCHAR(64) NOT NULL, joins_json JSON, fields_json JSON, filters_json JSON, file_format VARCHAR(16) NOT NULL, stats_enabled TINYINT(1) NOT NULL DEFAULT 0, sheet_split_by VARCHAR(64), visibility VARCHAR(16) NOT NULL DEFAULT 'private', owner_id BIGINT UNSIGNED NOT NULL, enabled TINYINT(1) NOT NULL DEFAULT 1, explain_json JSON, explain_score INT, last_validated_at DATETIME, created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP)",
|
||||
"CREATE TABLE IF NOT EXISTS export_jobs (id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY, template_id BIGINT UNSIGNED NOT NULL, status VARCHAR(16) NOT NULL, requested_by BIGINT UNSIGNED NOT NULL, permission_scope_json JSON, options_json JSON, row_estimate BIGINT, total_rows BIGINT, file_format VARCHAR(16) NOT NULL, started_at DATETIME, finished_at DATETIME, created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, INDEX idx_template_id (template_id), INDEX idx_status (status), INDEX idx_requested_by (requested_by))",
|
||||
"CREATE TABLE IF NOT EXISTS export_job_files (id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY, job_id BIGINT UNSIGNED NOT NULL, storage_uri VARCHAR(1024) NOT NULL, sheet_name VARCHAR(255), row_count BIGINT, size_bytes BIGINT, checksum VARCHAR(128), created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, INDEX idx_job_id (job_id))",
|
||||
"CREATE TABLE IF NOT EXISTS export_audits (id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY, actor_id BIGINT UNSIGNED NOT NULL, action VARCHAR(64) NOT NULL, entity_type VARCHAR(64) NOT NULL, entity_id BIGINT UNSIGNED NOT NULL, detail_json JSON, created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, INDEX idx_entity (entity_type, entity_id), INDEX idx_actor (actor_id))",
|
||||
}
|
||||
for _, s := range stmts {
|
||||
if _, err := db.Exec(s); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
comments := []string{
|
||||
"ALTER TABLE export_templates COMMENT='导出模板主表'",
|
||||
"ALTER TABLE export_templates MODIFY id BIGINT UNSIGNED AUTO_INCREMENT COMMENT '主键ID'",
|
||||
"ALTER TABLE export_templates MODIFY name VARCHAR(255) NOT NULL COMMENT '模板名称'",
|
||||
"ALTER TABLE export_templates MODIFY datasource VARCHAR(32) NOT NULL COMMENT '数据源标识'",
|
||||
"ALTER TABLE export_templates MODIFY main_table VARCHAR(64) NOT NULL COMMENT '主表名'",
|
||||
"ALTER TABLE export_templates MODIFY joins_json JSON COMMENT '关联表定义'",
|
||||
"ALTER TABLE export_templates MODIFY fields_json JSON COMMENT '字段选择'",
|
||||
"ALTER TABLE export_templates MODIFY filters_json JSON COMMENT '过滤条件'",
|
||||
"ALTER TABLE export_templates MODIFY file_format VARCHAR(16) NOT NULL COMMENT '文件格式'",
|
||||
"ALTER TABLE export_templates MODIFY stats_enabled TINYINT(1) NOT NULL COMMENT '统计开关'",
|
||||
"ALTER TABLE export_templates MODIFY sheet_split_by VARCHAR(64) COMMENT '多sheet拆分条件'",
|
||||
"ALTER TABLE export_templates MODIFY visibility VARCHAR(16) NOT NULL COMMENT '可见性'",
|
||||
"ALTER TABLE export_templates MODIFY owner_id BIGINT UNSIGNED NOT NULL COMMENT '所有者ID'",
|
||||
"ALTER TABLE export_templates MODIFY enabled TINYINT(1) NOT NULL COMMENT '启用状态'",
|
||||
"ALTER TABLE export_templates MODIFY explain_json JSON COMMENT 'EXPLAIN结果'",
|
||||
"ALTER TABLE export_templates MODIFY explain_score INT COMMENT 'EXPLAIN评分'",
|
||||
"ALTER TABLE export_templates MODIFY last_validated_at DATETIME COMMENT '最近校验时间'",
|
||||
"ALTER TABLE export_templates MODIFY created_at DATETIME NOT NULL COMMENT '创建时间'",
|
||||
"ALTER TABLE export_templates MODIFY updated_at DATETIME NOT NULL COMMENT '更新时间'",
|
||||
|
||||
"ALTER TABLE export_jobs COMMENT='导出任务表'",
|
||||
"ALTER TABLE export_jobs MODIFY id BIGINT UNSIGNED AUTO_INCREMENT COMMENT '主键ID'",
|
||||
"ALTER TABLE export_jobs MODIFY template_id BIGINT UNSIGNED NOT NULL COMMENT '模板ID'",
|
||||
"ALTER TABLE export_jobs MODIFY status VARCHAR(16) NOT NULL COMMENT '任务状态'",
|
||||
"ALTER TABLE export_jobs MODIFY requested_by BIGINT UNSIGNED NOT NULL COMMENT '请求人ID'",
|
||||
"ALTER TABLE export_jobs MODIFY permission_scope_json JSON COMMENT '权限范围'",
|
||||
"ALTER TABLE export_jobs MODIFY options_json JSON COMMENT '执行选项'",
|
||||
"ALTER TABLE export_jobs MODIFY row_estimate BIGINT COMMENT '行数估算'",
|
||||
"ALTER TABLE export_jobs MODIFY total_rows BIGINT COMMENT '实际行数'",
|
||||
"ALTER TABLE export_jobs MODIFY file_format VARCHAR(16) NOT NULL COMMENT '文件格式'",
|
||||
"ALTER TABLE export_jobs MODIFY started_at DATETIME COMMENT '开始时间'",
|
||||
"ALTER TABLE export_jobs MODIFY finished_at DATETIME COMMENT '完成时间'",
|
||||
"ALTER TABLE export_jobs MODIFY created_at DATETIME NOT NULL COMMENT '创建时间'",
|
||||
"ALTER TABLE export_jobs MODIFY updated_at DATETIME NOT NULL COMMENT '更新时间'",
|
||||
|
||||
"ALTER TABLE export_job_files COMMENT='导出文件记录'",
|
||||
"ALTER TABLE export_job_files MODIFY id BIGINT UNSIGNED AUTO_INCREMENT COMMENT '主键ID'",
|
||||
"ALTER TABLE export_job_files MODIFY job_id BIGINT UNSIGNED NOT NULL COMMENT '任务ID'",
|
||||
"ALTER TABLE export_job_files MODIFY storage_uri VARCHAR(1024) NOT NULL COMMENT '存储地址'",
|
||||
"ALTER TABLE export_job_files MODIFY sheet_name VARCHAR(255) COMMENT 'sheet名称'",
|
||||
"ALTER TABLE export_job_files MODIFY row_count BIGINT COMMENT '行数'",
|
||||
"ALTER TABLE export_job_files MODIFY size_bytes BIGINT COMMENT '文件大小'",
|
||||
"ALTER TABLE export_job_files MODIFY checksum VARCHAR(128) COMMENT '校验值'",
|
||||
"ALTER TABLE export_job_files MODIFY created_at DATETIME NOT NULL COMMENT '创建时间'",
|
||||
|
||||
"ALTER TABLE export_audits COMMENT='审计与变更记录'",
|
||||
"ALTER TABLE export_audits MODIFY id BIGINT UNSIGNED AUTO_INCREMENT COMMENT '主键ID'",
|
||||
"ALTER TABLE export_audits MODIFY actor_id BIGINT UNSIGNED NOT NULL COMMENT '操作者ID'",
|
||||
"ALTER TABLE export_audits MODIFY action VARCHAR(64) NOT NULL COMMENT '动作'",
|
||||
"ALTER TABLE export_audits MODIFY entity_type VARCHAR(64) NOT NULL COMMENT '对象类型'",
|
||||
"ALTER TABLE export_audits MODIFY entity_id BIGINT UNSIGNED NOT NULL COMMENT '对象ID'",
|
||||
"ALTER TABLE export_audits MODIFY detail_json JSON COMMENT '详情'",
|
||||
"ALTER TABLE export_audits MODIFY created_at DATETIME NOT NULL COMMENT '创建时间'",
|
||||
}
|
||||
for _, s := range comments {
|
||||
if _, err := db.Exec(s); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
optional := []string{
|
||||
"ALTER TABLE export_jobs ADD COLUMN explain_json JSON",
|
||||
"ALTER TABLE export_jobs ADD COLUMN explain_score INT",
|
||||
"ALTER TABLE export_jobs ADD COLUMN filters_json JSON",
|
||||
"ALTER TABLE export_job_files ADD COLUMN updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP",
|
||||
}
|
||||
for _, s := range optional {
|
||||
if _, err := db.Exec(s); err != nil {
|
||||
// ignore if column exists or syntax not supported
|
||||
continue
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
package models
|
||||
|
||||
import "time"
|
||||
|
||||
type Job struct {
|
||||
ID uint64
|
||||
TemplateID uint64
|
||||
Status string
|
||||
RequestedBy uint64
|
||||
PermissionScopeJSON []byte
|
||||
OptionsJSON []byte
|
||||
RowEstimate *int64
|
||||
TotalRows *int64
|
||||
FileFormat string
|
||||
StartedAt *time.Time
|
||||
FinishedAt *time.Time
|
||||
}
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
package models
|
||||
|
||||
type Template struct {
|
||||
ID uint64
|
||||
Name string
|
||||
Datasource string
|
||||
MainTable string
|
||||
JoinsJSON []byte
|
||||
FieldsJSON []byte
|
||||
FiltersJSON []byte
|
||||
FileFormat string
|
||||
StatsEnabled bool
|
||||
SheetSplitBy *string
|
||||
Visibility string
|
||||
OwnerID uint64
|
||||
Enabled bool
|
||||
ExplainJSON []byte
|
||||
ExplainScore *int
|
||||
}
|
||||
|
|
@ -0,0 +1,256 @@
|
|||
package repo
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"server/internal/exporter"
|
||||
"server/internal/logging"
|
||||
"time"
|
||||
)
|
||||
|
||||
type ExportQueryRepo struct{}
|
||||
|
||||
func NewExportRepo() *ExportQueryRepo { return &ExportQueryRepo{} }
|
||||
|
||||
func (r *ExportQueryRepo) Build(req exporter.BuildRequest, wl map[string]bool) (string, []interface{}, error) {
|
||||
return exporter.BuildSQL(req, wl)
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) Explain(db *sql.DB, q string, args []interface{}) (int, []string, error) {
|
||||
return exporter.EvaluateExplain(db, q, args)
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) Count(db *sql.DB, base string, args []interface{}) int64 {
|
||||
return exporter.CountRows(db, base, args)
|
||||
}
|
||||
|
||||
// Count by BuildRequest using filters-only joins and COUNT(DISTINCT main pk)
|
||||
func (r *ExportQueryRepo) CountByReq(db *sql.DB, req exporter.BuildRequest, wl map[string]bool) int64 {
|
||||
q, args, err := exporter.BuildCountSQL(req, wl)
|
||||
if err != nil {
|
||||
logging.JSON("ERROR", map[string]interface{}{"event": "build_count_sql_error", "error": err.Error()})
|
||||
return 0
|
||||
}
|
||||
var c int64
|
||||
row := db.QueryRow(q, args...)
|
||||
if err := row.Scan(&c); err != nil {
|
||||
logging.JSON("ERROR", map[string]interface{}{"event": "count_by_req_error", "error": err.Error(), "sql": q, "args": args})
|
||||
return 0
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) EstimateFast(db *sql.DB, ds, main string, filters map[string]interface{}) int64 {
|
||||
return exporter.CountRowsFast(db, ds, main, filters)
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) EstimateFastChunked(db *sql.DB, ds, main string, filters map[string]interface{}) int64 {
|
||||
return exporter.CountRowsFastChunked(db, ds, main, filters)
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) NewCursor(datasource, main string) *exporter.CursorSQL {
|
||||
return exporter.NewCursorSQL(datasource, main)
|
||||
}
|
||||
|
||||
type RowWriterFactory func() (exporter.RowWriter, error)
|
||||
type RowTransform func([]string) []string
|
||||
type RollCallback func(path string, size int64, partRows int64) error
|
||||
type ProgressCallback func(totalRows int64) error
|
||||
|
||||
func (r *ExportQueryRepo) StreamCursor(db *sql.DB, base string, args []interface{}, cur *exporter.CursorSQL, batch int, cols []string, newWriter RowWriterFactory, transform RowTransform, maxRowsPerFile int64, onRoll RollCallback, onProgress ProgressCallback) (int64, []string, error) {
|
||||
return exporter.StreamWithCursor(db, base, args, cur, batch, cols, func() (exporter.RowWriter, error) { return newWriter() }, func(vals []string) []string { return transform(vals) }, maxRowsPerFile, func(p string, sz int64, rows int64) error { return onRoll(p, sz, rows) }, func(total int64) error { return onProgress(total) })
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) ZipAndRecord(meta *sql.DB, jobID uint64, files []string, total int64) {
|
||||
if len(files) == 0 {
|
||||
return
|
||||
}
|
||||
zipPath, zipSize := exporter.ZipFiles(jobID, files)
|
||||
meta.Exec("INSERT INTO export_job_files (job_id, storage_uri, row_count, size_bytes, created_at, updated_at) VALUES (?,?,?,?,?,?)", jobID, zipPath, total, zipSize, time.Now(), time.Now())
|
||||
}
|
||||
|
||||
// Metadata and job helpers
|
||||
func (r *ExportQueryRepo) GetTemplateMeta(meta *sql.DB, tplID uint64) (string, string, []string, error) {
|
||||
var ds string
|
||||
var main string
|
||||
var fieldsJSON []byte
|
||||
row := meta.QueryRow("SELECT datasource, main_table, fields_json FROM export_templates WHERE id= ?", tplID)
|
||||
if err := row.Scan(&ds, &main, &fieldsJSON); err != nil {
|
||||
return "", "", nil, err
|
||||
}
|
||||
var fs []string
|
||||
_ = json.Unmarshal(fieldsJSON, &fs)
|
||||
return ds, main, fs, nil
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) GetJobFilters(meta *sql.DB, jobID uint64) (uint64, []byte, error) {
|
||||
var tplID uint64
|
||||
var filtersJSON []byte
|
||||
row := meta.QueryRow("SELECT template_id, filters_json FROM export_jobs WHERE id= ?", jobID)
|
||||
if err := row.Scan(&tplID, &filtersJSON); err != nil {
|
||||
return 0, nil, err
|
||||
}
|
||||
return tplID, filtersJSON, nil
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) InsertJob(meta *sql.DB, tplID, requestedBy, owner uint64, permission, filters, options map[string]interface{}, explain map[string]interface{}, explainScore int, rowEstimate int64, fileFormat string) (uint64, error) {
|
||||
ejSQL := "INSERT INTO export_jobs (template_id, status, requested_by, owner_id, permission_scope_json, filters_json, options_json, explain_json, explain_score, row_estimate, file_format, created_at, updated_at) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)"
|
||||
ejArgs := []interface{}{tplID, "queued", requestedBy, owner, toJSON(permission), toJSON(filters), toJSON(options), toJSON(explain), explainScore, rowEstimate, fileFormat, time.Now(), time.Now()}
|
||||
res, err := meta.Exec(ejSQL, ejArgs...)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
id, _ := res.LastInsertId()
|
||||
return uint64(id), nil
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) StartJob(meta *sql.DB, id uint64) {
|
||||
if _, err := meta.Exec("UPDATE export_jobs SET status=?, started_at=?, updated_at=? WHERE id= ?", "running", time.Now(), time.Now(), id); err != nil {
|
||||
logging.JSON("ERROR", map[string]interface{}{"event": "db_update_error", "action": "start_job", "job_id": id, "error": err.Error()})
|
||||
}
|
||||
}
|
||||
func (r *ExportQueryRepo) UpdateProgress(meta *sql.DB, id uint64, total int64) {
|
||||
if _, err := meta.Exec("UPDATE export_jobs SET total_rows=GREATEST(COALESCE(total_rows,0), ?), updated_at=?, status=CASE WHEN status='queued' THEN 'running' ELSE status END WHERE id= ?", total, time.Now(), id); err != nil {
|
||||
logging.JSON("ERROR", map[string]interface{}{"event": "db_update_error", "action": "update_progress", "job_id": id, "error": err.Error()})
|
||||
}
|
||||
logging.JSON("INFO", map[string]interface{}{"event": "progress_update", "job_id": id, "total_rows": total})
|
||||
}
|
||||
func (r *ExportQueryRepo) MarkFailed(meta *sql.DB, id uint64) {
|
||||
if _, err := meta.Exec("UPDATE export_jobs SET status=?, finished_at=? WHERE id= ?", "failed", time.Now(), id); err != nil {
|
||||
logging.JSON("ERROR", map[string]interface{}{"event": "db_update_error", "action": "mark_failed", "job_id": id, "error": err.Error()})
|
||||
}
|
||||
}
|
||||
func (r *ExportQueryRepo) MarkCompleted(meta *sql.DB, id uint64, total int64) {
|
||||
if _, err := meta.Exec("UPDATE export_jobs SET status=?, finished_at=?, total_rows=?, row_estimate=GREATEST(COALESCE(row_estimate,0), ?), updated_at=? WHERE id= ?", "completed", time.Now(), total, total, time.Now(), id); err != nil {
|
||||
logging.JSON("ERROR", map[string]interface{}{"event": "db_update_error", "action": "mark_completed", "job_id": id, "error": err.Error()})
|
||||
}
|
||||
}
|
||||
func (r *ExportQueryRepo) InsertJobFile(meta *sql.DB, id uint64, uri string, sheetName string, rowCount, size int64) {
|
||||
if _, err := meta.Exec("INSERT INTO export_job_files (job_id, storage_uri, sheet_name, row_count, size_bytes, created_at, updated_at) VALUES (?,?,?,?,?,?,?)", id, uri, sheetName, rowCount, size, time.Now(), time.Now()); err != nil {
|
||||
logging.JSON("ERROR", map[string]interface{}{"event": "db_insert_error", "action": "insert_job_file", "job_id": id, "error": err.Error(), "path": uri})
|
||||
}
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) UpdateRowEstimate(meta *sql.DB, id uint64, est int64) {
|
||||
if _, err := meta.Exec("UPDATE export_jobs SET row_estimate=?, updated_at=? WHERE id= ?", est, time.Now(), id); err != nil {
|
||||
logging.JSON("ERROR", map[string]interface{}{"event": "db_update_error", "action": "update_row_estimate", "job_id": id, "error": err.Error(), "row_estimate": est})
|
||||
}
|
||||
}
|
||||
|
||||
func toJSON(v interface{}) []byte {
|
||||
b, _ := json.Marshal(v)
|
||||
return b
|
||||
}
|
||||
|
||||
type JobDetail struct {
|
||||
ID uint64
|
||||
TemplateID uint64
|
||||
Status string
|
||||
RequestedBy uint64
|
||||
TotalRows sql.NullInt64
|
||||
FileFormat string
|
||||
StartedAt sql.NullTime
|
||||
FinishedAt sql.NullTime
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
ExplainScore sql.NullInt64
|
||||
ExplainJSON sql.NullString
|
||||
}
|
||||
|
||||
type JobFile struct {
|
||||
URI sql.NullString
|
||||
Sheet sql.NullString
|
||||
RowCount sql.NullInt64
|
||||
SizeBytes sql.NullInt64
|
||||
}
|
||||
|
||||
type JobListItem struct {
|
||||
ID uint64
|
||||
TemplateID uint64
|
||||
Status string
|
||||
RequestedBy uint64
|
||||
RowEstimate sql.NullInt64
|
||||
TotalRows sql.NullInt64
|
||||
FileFormat string
|
||||
CreatedAt sql.NullTime
|
||||
UpdatedAt sql.NullTime
|
||||
ExplainScore sql.NullInt64
|
||||
ExplainJSON sql.NullString
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) GetJob(meta *sql.DB, id string) (JobDetail, error) {
|
||||
row := meta.QueryRow("SELECT id, template_id, status, requested_by, total_rows, file_format, started_at, finished_at, created_at, updated_at, explain_score, explain_json FROM export_jobs WHERE id= ?", id)
|
||||
var d JobDetail
|
||||
err := row.Scan(&d.ID, &d.TemplateID, &d.Status, &d.RequestedBy, &d.TotalRows, &d.FileFormat, &d.StartedAt, &d.FinishedAt, &d.CreatedAt, &d.UpdatedAt, &d.ExplainScore, &d.ExplainJSON)
|
||||
return d, err
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) ListJobFiles(meta *sql.DB, jobID string) ([]JobFile, error) {
|
||||
rows, err := meta.Query("SELECT storage_uri, sheet_name, row_count, size_bytes FROM export_job_files WHERE job_id= ?", jobID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []JobFile{}
|
||||
for rows.Next() {
|
||||
var f JobFile
|
||||
rows.Scan(&f.URI, &f.Sheet, &f.RowCount, &f.SizeBytes)
|
||||
out = append(out, f)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) GetLatestFileURI(meta *sql.DB, jobID string) (string, error) {
|
||||
row := meta.QueryRow("SELECT storage_uri FROM export_job_files WHERE job_id=? ORDER BY id DESC LIMIT 1", jobID)
|
||||
var uri string
|
||||
err := row.Scan(&uri)
|
||||
return uri, err
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) CountJobs(meta *sql.DB, tplID uint64, owner string) int64 {
|
||||
var c int64
|
||||
if tplID > 0 {
|
||||
if owner != "" {
|
||||
_ = meta.QueryRow("SELECT COUNT(1) FROM export_jobs WHERE template_id = ? AND owner_id = ?", tplID, owner).Scan(&c)
|
||||
} else {
|
||||
_ = meta.QueryRow("SELECT COUNT(1) FROM export_jobs WHERE template_id = ?", tplID).Scan(&c)
|
||||
}
|
||||
} else {
|
||||
if owner != "" {
|
||||
_ = meta.QueryRow("SELECT COUNT(1) FROM export_jobs WHERE owner_id = ?", owner).Scan(&c)
|
||||
} else {
|
||||
_ = meta.QueryRow("SELECT COUNT(1) FROM export_jobs").Scan(&c)
|
||||
}
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
func (r *ExportQueryRepo) ListJobs(meta *sql.DB, tplID uint64, owner string, size, offset int) ([]JobListItem, error) {
|
||||
var rows *sql.Rows
|
||||
var err error
|
||||
if tplID > 0 {
|
||||
if owner != "" {
|
||||
rows, err = meta.Query("SELECT id, template_id, status, requested_by, row_estimate, total_rows, file_format, created_at, updated_at, explain_score, explain_json FROM export_jobs WHERE template_id = ? AND owner_id = ? ORDER BY id DESC LIMIT ? OFFSET ?", tplID, owner, size, offset)
|
||||
} else {
|
||||
rows, err = meta.Query("SELECT id, template_id, status, requested_by, row_estimate, total_rows, file_format, created_at, updated_at, explain_score, explain_json FROM export_jobs WHERE template_id = ? ORDER BY id DESC LIMIT ? OFFSET ?", tplID, size, offset)
|
||||
}
|
||||
} else {
|
||||
if owner != "" {
|
||||
rows, err = meta.Query("SELECT id, template_id, status, requested_by, row_estimate, total_rows, file_format, created_at, updated_at, explain_score, explain_json FROM export_jobs WHERE owner_id = ? ORDER BY id DESC LIMIT ? OFFSET ?", owner, size, offset)
|
||||
} else {
|
||||
rows, err = meta.Query("SELECT id, template_id, status, requested_by, row_estimate, total_rows, file_format, created_at, updated_at, explain_score, explain_json FROM export_jobs ORDER BY id DESC LIMIT ? OFFSET ?", size, offset)
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
items := []JobListItem{}
|
||||
for rows.Next() {
|
||||
var it JobListItem
|
||||
if err := rows.Scan(&it.ID, &it.TemplateID, &it.Status, &it.RequestedBy, &it.RowEstimate, &it.TotalRows, &it.FileFormat, &it.CreatedAt, &it.UpdatedAt, &it.ExplainScore, &it.ExplainJSON); err == nil {
|
||||
items = append(items, it)
|
||||
}
|
||||
}
|
||||
return items, nil
|
||||
}
|
||||
|
|
@ -0,0 +1,462 @@
|
|||
package schema
|
||||
|
||||
func AllWhitelist() map[string]bool {
|
||||
m := map[string]bool{
|
||||
"order.order_number": true,
|
||||
"order.key": true,
|
||||
"order.creator": true,
|
||||
"order.out_trade_no": true,
|
||||
"order.type": true,
|
||||
"order.status": true,
|
||||
"order.account": true,
|
||||
"order.product_id": true,
|
||||
"order.reseller_id": true,
|
||||
"order.plan_id": true,
|
||||
"order.key_batch_id": true,
|
||||
"order.code_batch_id": true,
|
||||
"order.pay_type": true,
|
||||
"order.pay_status": true,
|
||||
"order.use_coupon": true,
|
||||
"order.deliver_status": true,
|
||||
"order.expire_time": true,
|
||||
"order.recharge_time": true,
|
||||
"order.contract_price": true,
|
||||
"order.num": true,
|
||||
"order.total": true,
|
||||
"order.pay_amount": true,
|
||||
"order.create_time": true,
|
||||
"order.update_time": true,
|
||||
"order.card_code": true,
|
||||
"order.official_price": true,
|
||||
"order.merchant_name": true,
|
||||
"order.activity_name": true,
|
||||
"order.goods_name": true,
|
||||
"order.pay_time": true,
|
||||
"order.coupon_id": true,
|
||||
"order.discount_amount": true,
|
||||
"order.supplier_product_name": true,
|
||||
"order.is_inner": true,
|
||||
"order.icon": true,
|
||||
"order.cost_price": true,
|
||||
"order.success_num": true,
|
||||
"order.is_reset": true,
|
||||
"order.is_retry": true,
|
||||
"order.channel": true,
|
||||
"order.is_store": true,
|
||||
"order.trace_id": true,
|
||||
"order.out_order_no": true,
|
||||
"order.next_retry_time": true,
|
||||
"order.recharge_suc_time": true,
|
||||
"order.supplier_id": true,
|
||||
"order.supplier_product_id": true,
|
||||
"order.merchant_id": true,
|
||||
"order.goods_id": true,
|
||||
"order.activity_id": true,
|
||||
"order.key_batch_name": true,
|
||||
"order_detail.plan_title": true,
|
||||
"order_detail.order_number": true,
|
||||
"order_detail.reseller_name": true,
|
||||
"order_detail.product_name": true,
|
||||
"order_detail.show_url": true,
|
||||
"order_detail.official_price": true,
|
||||
"order_detail.cost_price": true,
|
||||
"order_detail.create_time": true,
|
||||
"order_detail.update_time": true,
|
||||
"order_cash.order_no": true,
|
||||
"order_cash.trade_no": true,
|
||||
"order_cash.wechat_detail_id": true,
|
||||
"order_cash.channel": true,
|
||||
"order_cash.denomination": true,
|
||||
"order_cash.account": true,
|
||||
"order_cash.receive_name": true,
|
||||
"order_cash.app_id": true,
|
||||
"order_cash.cash_activity_id": true,
|
||||
"order_cash.receive_status": true,
|
||||
"order_cash.receive_time": true,
|
||||
"order_cash.success_time": true,
|
||||
"order_cash.cash_packet_id": true,
|
||||
"order_cash.channel_order_id": true,
|
||||
"order_cash.pay_fund_order_id": true,
|
||||
"order_cash.cash_id": true,
|
||||
"order_cash.amount": true,
|
||||
"order_cash.activity_id": true,
|
||||
"order_cash.goods_id": true,
|
||||
"order_cash.merchant_id": true,
|
||||
"order_cash.supplier_id": true,
|
||||
"order_cash.user_id": true,
|
||||
"order_cash.status": true,
|
||||
"order_cash.expire_time": true,
|
||||
"order_cash.create_time": true,
|
||||
"order_cash.update_time": true,
|
||||
"order_cash.version": true,
|
||||
"order_cash.is_confirm": true,
|
||||
"order_voucher.channel": true,
|
||||
"order_voucher.channel_activity_id": true,
|
||||
"order_voucher.channel_voucher_id": true,
|
||||
"order_voucher.status": true,
|
||||
"order_voucher.receive_mode": true,
|
||||
"order_voucher.grant_time": true,
|
||||
"order_voucher.usage_time": true,
|
||||
"order_voucher.refund_time": true,
|
||||
"order_voucher.status_modify_time": true,
|
||||
"order_voucher.overdue_time": true,
|
||||
"order_voucher.refund_amount": true,
|
||||
"order_voucher.official_price": true,
|
||||
"order_voucher.out_biz_no": true,
|
||||
"order_voucher.account_no": true,
|
||||
"plan.id": true,
|
||||
"plan.title": true,
|
||||
"plan.status": true,
|
||||
"plan.begin_time": true,
|
||||
"plan.end_time": true,
|
||||
"key_batch.id": true,
|
||||
"key_batch.batch_name": true,
|
||||
"key_batch.bind_object": true,
|
||||
"key_batch.quantity": true,
|
||||
"key_batch.stock": true,
|
||||
"key_batch.begin_time": true,
|
||||
"key_batch.end_time": true,
|
||||
"code_batch.id": true,
|
||||
"code_batch.title": true,
|
||||
"code_batch.status": true,
|
||||
"code_batch.begin_time": true,
|
||||
"code_batch.end_time": true,
|
||||
"code_batch.quantity": true,
|
||||
"code_batch.usage": true,
|
||||
"code_batch.stock": true,
|
||||
"voucher.channel": true,
|
||||
"voucher.channel_activity_id": true,
|
||||
"voucher.price": true,
|
||||
"voucher.balance": true,
|
||||
"voucher.used_amount": true,
|
||||
"voucher.denomination": true,
|
||||
"voucher_batch.channel_activity_id": true,
|
||||
"voucher_batch.temp_no": true,
|
||||
"voucher_batch.provider": true,
|
||||
"voucher_batch.weight": true,
|
||||
"merchant_key_send.merchant_id": true,
|
||||
"merchant_key_send.out_biz_no": true,
|
||||
"merchant_key_send.key": true,
|
||||
"merchant_key_send.status": true,
|
||||
"merchant_key_send.usage_time": true,
|
||||
"merchant_key_send.create_time": true,
|
||||
"order_digit.order_no": true,
|
||||
"order_digit.card_no": true,
|
||||
"order_digit.account": true,
|
||||
"order_digit.goods_id": true,
|
||||
"order_digit.merchant_id": true,
|
||||
"order_digit.supplier_id": true,
|
||||
"order_digit.activity_id": true,
|
||||
"order_digit.user_id": true,
|
||||
"order_digit.success_time": true,
|
||||
"order_digit.supplier_product_no": true,
|
||||
"order_digit.order_type": true,
|
||||
"order_digit.end_time": true,
|
||||
"order_digit.create_time": true,
|
||||
"order_digit.update_time": true,
|
||||
"order_digit.code": true,
|
||||
"order_digit.sms_channel": true,
|
||||
"goods_voucher_batch.channel_batch_no": true,
|
||||
"goods_voucher_batch.voucher_subject_id": true,
|
||||
"goods_voucher_batch.id": true,
|
||||
"goods_voucher_batch.goods_voucher_id": true,
|
||||
"goods_voucher_batch.supplier_id": true,
|
||||
"goods_voucher_batch.temp_no": true,
|
||||
"goods_voucher_batch.index": true,
|
||||
"goods_voucher_batch.create_time": true,
|
||||
"goods_voucher_batch.update_time": true,
|
||||
"goods_voucher_subject_config.id": true,
|
||||
"goods_voucher_subject_config.name": true,
|
||||
"goods_voucher_subject_config.type": true,
|
||||
"goods_voucher_subject_config.create_time": true,
|
||||
"merchant.id": true,
|
||||
"merchant.name": true,
|
||||
"merchant.user_id": true,
|
||||
"merchant.merchant_no": true,
|
||||
"merchant.subject": true,
|
||||
"merchant.third_party": true,
|
||||
"merchant.status": true,
|
||||
"merchant.balance": true,
|
||||
"merchant.total_consumption": true,
|
||||
"merchant.contact_name": true,
|
||||
"merchant.contact_phone": true,
|
||||
"merchant.contact_email": true,
|
||||
"merchant.create_time": true,
|
||||
"merchant.update_time": true,
|
||||
"activity.id": true,
|
||||
"activity.name": true,
|
||||
"activity.user_id": true,
|
||||
"activity.merchant_id": true,
|
||||
"activity.user_name": true,
|
||||
"activity.activity_no": true,
|
||||
"activity.status": true,
|
||||
"activity.key_total_num": true,
|
||||
"activity.key_generate_num": true,
|
||||
"activity.key_usable_num": true,
|
||||
"activity.domain_url": true,
|
||||
"activity.theme_login_id": true,
|
||||
"activity.theme_list_id": true,
|
||||
"activity.theme_verify_id": true,
|
||||
"activity.settlement_type": true,
|
||||
"activity.key_expire_type": true,
|
||||
"activity.key_valid_day": true,
|
||||
"activity.key_begin_time": true,
|
||||
"activity.key_end_time": true,
|
||||
"activity.key_style": true,
|
||||
"activity.begin_time": true,
|
||||
"activity.end_time": true,
|
||||
"activity.is_retry": true,
|
||||
"activity.create_time": true,
|
||||
"activity.update_time": true,
|
||||
"activity.discard_time": true,
|
||||
"activity.delete_time": true,
|
||||
"activity.auto_charge": true,
|
||||
"activity.stock": true,
|
||||
"activity.approval_trade_no": true,
|
||||
"activity.amount": true,
|
||||
"activity.channels": true,
|
||||
"activity.key_begin": true,
|
||||
"activity.key_end": true,
|
||||
"activity.key_unit": true,
|
||||
"activity.key_pay_button_text": true,
|
||||
"activity.goods_pay_button_text": true,
|
||||
"activity.is_open_db_transaction": true,
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func AllLabels() map[string]string {
|
||||
return map[string]string{
|
||||
"order.order_number": "订单编号",
|
||||
"order.key": "KEY",
|
||||
"order.creator": "创建者ID",
|
||||
"order.out_trade_no": "支付流水号",
|
||||
"order.type": "订单类型",
|
||||
"order.status": "订单状态",
|
||||
"order.account": "账号",
|
||||
"order.product_id": "商品ID",
|
||||
"order.reseller_id": "分销商ID",
|
||||
"order.plan_id": "计划ID",
|
||||
"order.key_batch_id": "KEY批次ID",
|
||||
"order.code_batch_id": "兑换批次ID",
|
||||
"order.pay_type": "支付方式",
|
||||
"order.pay_status": "支付状态",
|
||||
"order.use_coupon": "是否使用优惠券",
|
||||
"order.deliver_status": "投递状态",
|
||||
"order.expire_time": "过期处理时间",
|
||||
"order.recharge_time": "充值时间",
|
||||
"order.contract_price": "合同单价",
|
||||
"order.num": "数量",
|
||||
"order.total": "总金额",
|
||||
"order.pay_amount": "支付金额",
|
||||
"order.create_time": "创建时间",
|
||||
"order.update_time": "更新时间",
|
||||
"order.official_price": "官方价",
|
||||
"order.merchant_name": "分销商名称",
|
||||
"order.activity_name": "活动名称",
|
||||
"order.goods_name": "商品名称",
|
||||
"order.pay_time": "支付时间",
|
||||
"order.coupon_id": "优惠券ID",
|
||||
"order.discount_amount": "优惠金额",
|
||||
"order.card_code": "卡密(脱敏)",
|
||||
"order.supplier_product_name": "供应商产品名称",
|
||||
"order.is_inner": "内部供应商订单",
|
||||
"order.icon": "订单图片",
|
||||
"order.cost_price": "成本价",
|
||||
"order.success_num": "到账数量",
|
||||
"order.is_reset": "是否重置",
|
||||
"order.is_retry": "是否重试",
|
||||
"order.channel": "支付渠道",
|
||||
"order.is_store": "是否退还库存",
|
||||
"order.trace_id": "TraceID",
|
||||
"order.out_order_no": "外部订单号",
|
||||
"order.next_retry_time": "下次重试时间",
|
||||
"order.recharge_suc_time": "充值成功时间",
|
||||
"order.supplier_id": "供应商ID",
|
||||
"order.supplier_product_id": "供应商产品ID",
|
||||
"order.merchant_id": "分销商ID",
|
||||
"order.goods_id": "商品ID",
|
||||
"order.activity_id": "活动ID",
|
||||
"order.key_batch_name": "key批次名称",
|
||||
"order_detail.plan_title": "计划标题",
|
||||
"order_detail.order_number": "订单编号",
|
||||
"order_detail.reseller_name": "分销商名称",
|
||||
"order_detail.product_name": "商品名称",
|
||||
"order_detail.show_url": "商品图片URL",
|
||||
"order_detail.official_price": "官方价",
|
||||
"order_detail.cost_price": "成本价",
|
||||
"order_detail.create_time": "创建时间",
|
||||
"order_detail.update_time": "更新时间",
|
||||
"order_cash.order_no": "订单号",
|
||||
"order_cash.trade_no": "交易号",
|
||||
"order_cash.wechat_detail_id": "微信明细单号",
|
||||
"order_cash.channel": "渠道",
|
||||
"order_cash.denomination": "红包面额",
|
||||
"order_cash.account": "领取账号",
|
||||
"order_cash.receive_name": "真实姓名",
|
||||
"order_cash.app_id": "转账AppID",
|
||||
"order_cash.cash_activity_id": "红包批次号",
|
||||
"order_cash.receive_status": "领取状态",
|
||||
"order_cash.receive_time": "拆红包时间",
|
||||
"order_cash.success_time": "成功时间",
|
||||
"order_cash.cash_packet_id": "红包ID",
|
||||
"order_cash.channel_order_id": "渠道订单号",
|
||||
"order_cash.pay_fund_order_id": "资金订单号",
|
||||
"order_cash.cash_id": "红包规则ID",
|
||||
"order_cash.amount": "红包额度",
|
||||
"order_cash.activity_id": "活动ID",
|
||||
"order_cash.goods_id": "商品ID",
|
||||
"order_cash.merchant_id": "分销商ID",
|
||||
"order_cash.supplier_id": "供应商ID",
|
||||
"order_cash.user_id": "创建者ID",
|
||||
"order_cash.status": "状态",
|
||||
"order_cash.expire_time": "过期时间",
|
||||
"order_cash.create_time": "创建时间",
|
||||
"order_cash.update_time": "更新时间",
|
||||
"order_cash.version": "版本",
|
||||
"order_cash.is_confirm": "是否确认",
|
||||
"order_voucher.channel": "渠道",
|
||||
"order_voucher.channel_activity_id": "渠道立减金批次",
|
||||
"order_voucher.channel_voucher_id": "渠道立减金ID",
|
||||
"order_voucher.status": "状态",
|
||||
"order_voucher.receive_mode": "领取方式",
|
||||
"order_voucher.grant_time": "领取时间",
|
||||
"order_voucher.usage_time": "核销时间",
|
||||
"order_voucher.refund_time": "退款时间",
|
||||
"order_voucher.status_modify_time": "状态更新时间",
|
||||
"order_voucher.overdue_time": "过期时间",
|
||||
"order_voucher.refund_amount": "退款金额",
|
||||
"order_voucher.official_price": "官方价",
|
||||
"order_voucher.out_biz_no": "外部业务号",
|
||||
"order_voucher.account_no": "账户号",
|
||||
"plan.id": "计划ID",
|
||||
"plan.title": "计划标题",
|
||||
"plan.status": "状态",
|
||||
"plan.begin_time": "开始时间",
|
||||
"plan.end_time": "结束时间",
|
||||
"key_batch.id": "批次ID",
|
||||
"key_batch.batch_name": "批次名称",
|
||||
"key_batch.bind_object": "绑定对象",
|
||||
"key_batch.quantity": "发放数量",
|
||||
"key_batch.stock": "剩余库存",
|
||||
"key_batch.begin_time": "开始时间",
|
||||
"key_batch.end_time": "结束时间",
|
||||
"code_batch.id": "兑换批次ID",
|
||||
"code_batch.title": "标题",
|
||||
"code_batch.status": "状态",
|
||||
"code_batch.begin_time": "开始时间",
|
||||
"code_batch.end_time": "结束时间",
|
||||
"code_batch.quantity": "数量",
|
||||
"code_batch.usage": "使用数",
|
||||
"code_batch.stock": "库存",
|
||||
"voucher.channel": "渠道",
|
||||
"voucher.channel_activity_id": "渠道批次号",
|
||||
"voucher.price": "合同单价",
|
||||
"voucher.balance": "剩余额度",
|
||||
"voucher.used_amount": "已用额度",
|
||||
"voucher.denomination": "面额",
|
||||
"voucher_batch.channel_activity_id": "渠道批次号",
|
||||
"voucher_batch.temp_no": "模板编号",
|
||||
"voucher_batch.provider": "服务商",
|
||||
"voucher_batch.weight": "权重",
|
||||
"merchant_key_send.merchant_id": "商户ID",
|
||||
"merchant_key_send.out_biz_no": "商户业务号",
|
||||
"merchant_key_send.key": "券码",
|
||||
"merchant_key_send.status": "状态",
|
||||
"merchant_key_send.usage_time": "核销时间",
|
||||
"merchant_key_send.create_time": "创建时间",
|
||||
"order_digit.order_no": "订单号",
|
||||
"order_digit.card_no": "卡号",
|
||||
"order_digit.account": "充值账号",
|
||||
"order_digit.goods_id": "商品ID",
|
||||
"order_digit.merchant_id": "分销商ID",
|
||||
"order_digit.supplier_id": "供应商ID",
|
||||
"order_digit.activity_id": "活动ID",
|
||||
"order_digit.user_id": "创建者ID",
|
||||
"order_digit.success_time": "到账时间",
|
||||
"order_digit.supplier_product_no": "供应商产品编码",
|
||||
"order_digit.order_type": "订单类型",
|
||||
"order_digit.end_time": "卡密有效期",
|
||||
"order_digit.create_time": "创建时间",
|
||||
"order_digit.update_time": "更新时间",
|
||||
"order_digit.code": "验证码",
|
||||
"order_digit.sms_channel": "短信渠道",
|
||||
"goods_voucher_batch.channel_batch_no": "渠道批次号",
|
||||
"goods_voucher_batch.voucher_subject_id": "主体配置ID",
|
||||
"goods_voucher_batch.id": "ID",
|
||||
"goods_voucher_batch.goods_voucher_id": "立减金ID",
|
||||
"goods_voucher_batch.supplier_id": "供应商ID",
|
||||
"goods_voucher_batch.temp_no": "模板编号",
|
||||
"goods_voucher_batch.index": "权重",
|
||||
"goods_voucher_batch.create_time": "创建时间",
|
||||
"goods_voucher_batch.update_time": "更新时间",
|
||||
"goods_voucher_subject_config.id": "主体配置ID",
|
||||
"goods_voucher_subject_config.name": "主体名称",
|
||||
"goods_voucher_subject_config.type": "主体类型",
|
||||
"goods_voucher_subject_config.create_time": "创建时间",
|
||||
"merchant.id": "客户ID",
|
||||
"merchant.name": "客户名称",
|
||||
"merchant.user_id": "用户中心ID",
|
||||
"merchant.merchant_no": "商户编码",
|
||||
"merchant.subject": "客户主体",
|
||||
"merchant.third_party": "来源类型",
|
||||
"merchant.status": "状态",
|
||||
"merchant.balance": "客户余额",
|
||||
"merchant.total_consumption": "累计消费",
|
||||
"merchant.contact_name": "联系人名称",
|
||||
"merchant.contact_phone": "联系人电话",
|
||||
"merchant.contact_email": "联系人Email",
|
||||
"merchant.create_time": "创建时间",
|
||||
"merchant.update_time": "编辑时间",
|
||||
"activity.id": "活动ID",
|
||||
"activity.name": "活动名称",
|
||||
"activity.user_id": "创建者ID",
|
||||
"activity.merchant_id": "客户ID",
|
||||
"activity.user_name": "创建者名称",
|
||||
"activity.activity_no": "活动编号",
|
||||
"activity.status": "状态",
|
||||
"activity.key_total_num": "Key码总量",
|
||||
"activity.key_generate_num": "Key码已生成数量",
|
||||
"activity.key_usable_num": "Key可使用次数",
|
||||
"activity.domain_url": "域名",
|
||||
"activity.theme_login_id": "登录模版ID",
|
||||
"activity.theme_list_id": "列表模版ID",
|
||||
"activity.theme_verify_id": "验证模版ID",
|
||||
"activity.settlement_type": "结算方式",
|
||||
"activity.key_expire_type": "Key有效期类型",
|
||||
"activity.key_valid_day": "有效天数",
|
||||
"activity.key_begin_time": "Key有效开始时间",
|
||||
"activity.key_end_time": "Key有效结束时间",
|
||||
"activity.key_style": "Key样式",
|
||||
"activity.begin_time": "开始时间",
|
||||
"activity.end_time": "结束时间",
|
||||
"activity.is_retry": "是否自动重试",
|
||||
"activity.create_time": "创建时间",
|
||||
"activity.update_time": "修改时间",
|
||||
"activity.discard_time": "作废时间",
|
||||
"activity.delete_time": "删除时间",
|
||||
"activity.auto_charge": "是否充值到账",
|
||||
"activity.stock": "已使用库存",
|
||||
"activity.approval_trade_no": "审批交易号",
|
||||
"activity.amount": "支付金额",
|
||||
"activity.channels": "支付渠道",
|
||||
"activity.key_begin": "开始月份",
|
||||
"activity.key_end": "截止月份",
|
||||
"activity.key_unit": "时间单位",
|
||||
"activity.key_pay_button_text": "Key支付按钮文本",
|
||||
"activity.goods_pay_button_text": "商品支付按钮文本",
|
||||
"activity.is_open_db_transaction": "是否开启事务",
|
||||
// removed bank_tag: not present in current YMT activity schema
|
||||
}
|
||||
}
|
||||
|
||||
func RecommendedDefaultFields(ds string) []string {
|
||||
if ds == "ymt" {
|
||||
return []string{
|
||||
"order.order_number", "order.creator", "order.out_trade_no", "order.type", "order.status", "order.contract_price", "order.num", "order.pay_amount", "order.create_time",
|
||||
}
|
||||
}
|
||||
return []string{
|
||||
"order.order_number", "order.creator", "order.out_trade_no", "order.type", "order.status", "order.contract_price", "order.num", "order.total", "order.pay_amount", "order.create_time",
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
package schema
|
||||
|
||||
type marketingSchema struct{}
|
||||
|
||||
func (marketingSchema) TableName(t string) string { return t }
|
||||
|
||||
func (marketingSchema) MapField(t, f string) (string, bool) { return f, true }
|
||||
|
||||
func (marketingSchema) BuildJoins(need map[string]bool, main string) []string {
|
||||
out := []string{}
|
||||
if need["order_detail"] {
|
||||
out = append(out, " LEFT JOIN `order_detail` ON `order_detail`.order_number = `order`.order_number")
|
||||
}
|
||||
if need["order_cash"] {
|
||||
out = append(out, " LEFT JOIN `order_cash` ON `order_cash`.order_number = `order`.order_number")
|
||||
}
|
||||
if need["order_voucher"] {
|
||||
out = append(out, " LEFT JOIN `order_voucher` ON `order_voucher`.order_number = `order`.order_number")
|
||||
}
|
||||
if need["plan"] || need["key_batch"] {
|
||||
out = append(out, " LEFT JOIN `plan` ON `plan`.id = `order`.plan_id")
|
||||
}
|
||||
if need["key_batch"] {
|
||||
out = append(out, " LEFT JOIN `key_batch` ON `key_batch`.plan_id = `plan`.id")
|
||||
}
|
||||
if need["code_batch"] {
|
||||
out = append(out, " LEFT JOIN `code_batch` ON `code_batch`.key_batch_id = `key_batch`.id")
|
||||
}
|
||||
if need["voucher"] {
|
||||
out = append(out, " LEFT JOIN `voucher` ON `voucher`.channel_activity_id = `order_voucher`.channel_activity_id")
|
||||
}
|
||||
if need["voucher_batch"] {
|
||||
out = append(out, " LEFT JOIN `voucher_batch` ON `voucher_batch`.voucher_id = `voucher`.id")
|
||||
}
|
||||
if need["merchant_key_send"] {
|
||||
out = append(out, " LEFT JOIN `merchant_key_send` ON `order`.`key` = `merchant_key_send`.key")
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (marketingSchema) FilterColumn(key string) (string, string, bool) {
|
||||
switch key {
|
||||
case "creator_in": return "order", "creator", true
|
||||
case "create_time_between": return "order", "create_time", true
|
||||
case "type_eq": return "order", "type", true
|
||||
case "out_trade_no_eq": return "order", "out_trade_no", true
|
||||
case "account_eq": return "order", "account", true
|
||||
case "plan_id_eq": return "order", "plan_id", true
|
||||
case "key_batch_id_eq": return "order", "key_batch_id", true
|
||||
case "product_id_eq": return "order", "product_id", true
|
||||
case "reseller_id_eq": return "order", "reseller_id", true
|
||||
case "code_batch_id_eq": return "order", "code_batch_id", true
|
||||
case "order_cash_cash_activity_id_eq": return "order_cash", "cash_activity_id", true
|
||||
case "order_voucher_channel_activity_id_eq": return "order_voucher", "channel_activity_id", true
|
||||
case "voucher_batch_channel_activity_id_eq": return "voucher_batch", "channel_activity_id", true
|
||||
case "merchant_out_biz_no_eq": return "merchant_key_send", "out_biz_no", true
|
||||
default:
|
||||
return "", "", false
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
package schema
|
||||
|
||||
type Schema interface {
|
||||
TableName(string) string
|
||||
MapField(string, string) (string, bool)
|
||||
BuildJoins(map[string]bool, string) []string
|
||||
FilterColumn(string) (string, string, bool)
|
||||
}
|
||||
|
||||
func Get(datasource string, main string) Schema {
|
||||
if datasource == "ymt" || main == "order_info" {
|
||||
return ymtSchema{}
|
||||
}
|
||||
return marketingSchema{}
|
||||
}
|
||||
|
|
@ -0,0 +1,83 @@
|
|||
package schema
|
||||
|
||||
type ymtSchema struct{}
|
||||
|
||||
func (ymtSchema) TableName(t string) string {
|
||||
if t == "order" {
|
||||
return "order_info"
|
||||
}
|
||||
return t
|
||||
}
|
||||
|
||||
func (s ymtSchema) MapField(t, f string) (string, bool) {
|
||||
if t == "order" {
|
||||
switch f {
|
||||
case "order_number": return "order_no", true
|
||||
case "key": return "key_code", true
|
||||
case "creator": return "user_id", true
|
||||
case "out_trade_no": return "out_order_no", true
|
||||
case "plan_id": return "activity_id", true
|
||||
case "reseller_id": return "merchant_id", true
|
||||
case "product_id": return "goods_id", true
|
||||
case "pay_amount": return "pay_price", true
|
||||
case "key_batch_id": return "key_batch_name", true
|
||||
default:
|
||||
return f, true
|
||||
}
|
||||
}
|
||||
if t == "order_voucher" {
|
||||
switch f {
|
||||
case "channel_activity_id": return "channel_batch_no", true
|
||||
case "overdue_time": return "expire_time", true
|
||||
case "account_no": return "account", true
|
||||
default:
|
||||
return f, true
|
||||
}
|
||||
}
|
||||
return f, true
|
||||
}
|
||||
|
||||
func (s ymtSchema) BuildJoins(need map[string]bool, main string) []string {
|
||||
out := []string{}
|
||||
if need["order_cash"] {
|
||||
out = append(out, " LEFT JOIN `order_cash` ON `order_cash`.order_no = `order_info`.order_no")
|
||||
}
|
||||
if need["order_voucher"] {
|
||||
out = append(out, " LEFT JOIN `order_voucher` ON `order_voucher`.order_no = `order_info`.order_no")
|
||||
}
|
||||
if need["order_digit"] {
|
||||
out = append(out, " LEFT JOIN `order_digit` ON `order_digit`.order_no = `order_info`.order_no")
|
||||
}
|
||||
if need["goods_voucher_batch"] {
|
||||
out = append(out, " LEFT JOIN `goods_voucher_batch` ON `goods_voucher_batch`.channel_batch_no = `order_voucher`.channel_batch_no")
|
||||
}
|
||||
if need["goods_voucher_subject_config"] {
|
||||
out = append(out, " LEFT JOIN `goods_voucher_subject_config` ON `goods_voucher_subject_config`.id = `goods_voucher_batch`.voucher_subject_id")
|
||||
}
|
||||
if need["merchant"] {
|
||||
out = append(out, " LEFT JOIN `merchant` ON `merchant`.id = `order_info`.merchant_id")
|
||||
}
|
||||
if need["activity"] {
|
||||
out = append(out, " LEFT JOIN `activity` ON `activity`.id = `order_info`.activity_id")
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (s ymtSchema) FilterColumn(key string) (string, string, bool) {
|
||||
switch key {
|
||||
case "creator_in": return "order", "user_id", true
|
||||
case "create_time_between": return "order", "create_time", true
|
||||
case "type_eq": return "order", "type", true
|
||||
case "out_trade_no_eq": return "order", "out_order_no", true
|
||||
case "account_eq": return "order", "account", true
|
||||
case "plan_id_eq": return "order", "activity_id", true
|
||||
case "key_batch_id_eq": return "order", "key_batch_name", true
|
||||
case "product_id_eq": return "order", "goods_id", true
|
||||
case "reseller_id_eq": return "order", "merchant_id", true
|
||||
case "code_batch_id_eq": return "order", "supplier_product_id", true
|
||||
case "order_cash_cash_activity_id_eq": return "order_cash", "activity_id", true
|
||||
case "order_voucher_channel_activity_id_eq": return "order_voucher", "channel_batch_no", true
|
||||
default:
|
||||
return "", "", false
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
package ymtcrypto
|
||||
|
||||
import (
|
||||
"crypto/cipher"
|
||||
"encoding/base64"
|
||||
"errors"
|
||||
sm4 "github.com/tjfoc/gmsm/sm4"
|
||||
)
|
||||
|
||||
func SM4Decrypt(encrypted, encryptKey string) (string, error) {
|
||||
if encrypted == "" {
|
||||
return "", nil
|
||||
}
|
||||
d, err := base64.StdEncoding.DecodeString(encryptKey)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(d) != 16 {
|
||||
return "", errors.New("invalid sm4 key length")
|
||||
}
|
||||
cipherBlock, err := sm4.NewCipher(d)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
blockSize := cipherBlock.BlockSize()
|
||||
iv := make([]byte, blockSize)
|
||||
for i := 0; i < blockSize; i++ { iv[i] = 0 }
|
||||
cipherText, err := base64.StdEncoding.DecodeString(encrypted)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(cipherText)%blockSize != 0 {
|
||||
return "", errors.New("invalid sm4 ciphertext size")
|
||||
}
|
||||
plainText := make([]byte, len(cipherText))
|
||||
blockMode := cipher.NewCBCDecrypter(cipherBlock, iv)
|
||||
blockMode.CryptBlocks(plainText, cipherText)
|
||||
if len(plainText) == 0 {
|
||||
return "", nil
|
||||
}
|
||||
padding := int(plainText[len(plainText)-1])
|
||||
if padding <= 0 || padding > len(plainText) {
|
||||
return "", errors.New("invalid padding")
|
||||
}
|
||||
buff := plainText[:len(plainText)-padding]
|
||||
return string(buff), nil
|
||||
}
|
||||
|
||||
Binary file not shown.
|
|
@ -0,0 +1,312 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>MarketingSystemDataTool</title>
|
||||
<link rel="stylesheet" href="./vendor/element-plus.min.css">
|
||||
<link rel="stylesheet" href="./styles.css">
|
||||
</head>
|
||||
<body>
|
||||
<div id="app">
|
||||
<el-container>
|
||||
<el-header height="56px">
|
||||
<el-row align="middle" justify="space-between">
|
||||
<el-col :span="24">
|
||||
<div class="title">导出工具</div>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</el-header>
|
||||
<el-main>
|
||||
<el-row :gutter="16">
|
||||
<el-col :span="24">
|
||||
<el-card>
|
||||
<template #header>
|
||||
<div style="display:flex;align-items:center;gap:8px">
|
||||
<span>导出模版</span>
|
||||
<el-button type="primary" size="small" @click="createVisible=true">新增模板</el-button>
|
||||
</div>
|
||||
</template>
|
||||
<el-table :data="templates" size="small" stripe>
|
||||
<el-table-column prop="id" label="ID" width="80"></el-table-column>
|
||||
<el-table-column prop="name" label="名称"></el-table-column>
|
||||
<el-table-column label="数据源" width="120">
|
||||
<template #default="scope">{{ dsLabel(scope.row.datasource) }}</template>
|
||||
</el-table-column>
|
||||
<el-table-column prop="file_format" label="格式" width="100"></el-table-column>
|
||||
<el-table-column prop="field_count" label="字段数" width="100"></el-table-column>
|
||||
|
||||
<el-table-column label="操作" width="260">
|
||||
<template #default="scope">
|
||||
<el-button size="small" type="primary" @click="openExport(scope.row)">执行</el-button>
|
||||
<el-button size="small" @click="openJobs(scope.row)">任务</el-button>
|
||||
<el-button
|
||||
v-if="(!hasUserId) || (hasUserId && Number(scope.row.owner_id)===currentUserId)"
|
||||
size="small"
|
||||
@click="openEdit(scope.row)">编辑</el-button>
|
||||
<el-button
|
||||
v-if="(!hasUserId) || (Number(scope.row.owner_id)!==0 && Number(scope.row.owner_id)===currentUserId)"
|
||||
size="small"
|
||||
type="danger"
|
||||
@click="removeTemplate(scope.row.id)">删除</el-button>
|
||||
</template>
|
||||
</el-table-column>
|
||||
</el-table>
|
||||
</el-card>
|
||||
</el-col>
|
||||
<el-col :span="24">
|
||||
<el-dialog v-model="jobsVisible" :title="'导出任务（模板 '+ (jobsTplId||'') +'）'" width="1000px">
|
||||
<el-table :data="jobs" size="small" stripe row-key="id">
|
||||
<el-table-column prop="id" label="ID"></el-table-column>
|
||||
<el-table-column label="校验状态">
|
||||
<template #default="scope">{{ scope.row.eval_status || '评估中' }}</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="进度">
|
||||
<template #default="scope">{{ jobPercent(scope.row) }}</template>
|
||||
</el-table-column>
|
||||
|
||||
<el-table-column label="行数">
|
||||
<template #default="scope">
|
||||
{{ Number(scope.row.row_estimate||0) > 0 ? Number(scope.row.row_estimate||0) : '评估中' }}
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column prop="total_rows" label="已写行数"></el-table-column>
|
||||
<el-table-column prop="file_format" label="格式"></el-table-column>
|
||||
<el-table-column label="创建时间">
|
||||
<template #default="scope">{{ fmtDT(new Date(scope.row.created_at)) }}</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="操作" width="200">
|
||||
<template #default="scope">
|
||||
<el-button size="small" @click="openSQL(scope.row.id)">分析</el-button>
|
||||
<el-button v-if="scope.row.status==='completed' && Number(scope.row.total_rows)>0" size="small" type="success" @click="download(scope.row.id)">下载</el-button>
|
||||
<el-button v-else-if="scope.row.status==='completed'" size="small" disabled>无数据</el-button>
|
||||
</template>
|
||||
</el-table-column>
|
||||
</el-table>
|
||||
|
||||
<div style="display:flex;justify-content:flex-end;margin-top:8px">
|
||||
<el-pagination background layout="prev, pager, next, total" :total="jobsTotal" :page-size="jobsPageSize" v-model:currentPage="jobsPage" @current-change="loadJobs" />
|
||||
</div>
|
||||
<template #footer>
|
||||
<el-button @click="closeJobs">关闭</el-button>
|
||||
</template>
|
||||
</el-dialog>
|
||||
<el-dialog v-model="sqlVisible" title="生成SQL" width="800px">
|
||||
<div style="max-height:50vh;overflow:auto"><pre style="white-space:pre-wrap">{{ sqlText }}</pre></div>
|
||||
<template #footer>
|
||||
<div style="display:flex; align-items:center; justify-content:space-between; gap:12px; width:100%">
|
||||
<div style="flex:1; min-width:0; text-align:left; color:#606266">
|
||||
<el-text v-if="sqlExplainDesc" type="info" truncated>{{ sqlExplainDesc }}</el-text>
|
||||
<el-text v-else type="info">评估中或暂无分析结果</el-text>
|
||||
</div>
|
||||
<div style="display:flex; gap:8px">
|
||||
<el-button @click="sqlVisible=false">关闭</el-button>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
</el-dialog>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</el-main>
|
||||
</el-container>
|
||||
<el-dialog v-model="createVisible" title="新增模板" :width="createWidth">
|
||||
<el-form ref="createFormRef" :model="form" :rules="createRules" label-width="110px" status-icon>
|
||||
<el-form-item label="模板名称" required show-message prop="name"><el-input v-model="form.name" placeholder="模板名称" /></el-form-item>
|
||||
<el-form-item label="数据源" required show-message>
|
||||
<el-select v-model="form.datasource" placeholder="选择" :teleported="false">
|
||||
<el-option v-for="opt in datasourceOptions" :key="opt.value" :label="opt.label" :value="opt.value" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
<el-form-item label="导出场景" required show-message>
|
||||
<el-select v-model="form.main_table" placeholder="订单数据">
|
||||
<el-option v-for="opt in sceneOptions" :key="opt.value" :label="opt.label" :value="opt.value" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
<el-form-item label="订单类型" required show-message prop="orderType">
|
||||
<el-radio-group v-model="form.orderType">
|
||||
<el-radio v-for="opt in orderTypeOptionsFor(form.datasource)" :key="opt.value" :label="opt.value">{{ opt.label }}</el-radio>
|
||||
</el-radio-group>
|
||||
</el-form-item>
|
||||
<el-form-item label="字段选择" required show-message prop="fieldsSel">
|
||||
<div ref="createCascaderRoot">
|
||||
<el-cascader
|
||||
ref="fieldsCascader"
|
||||
v-model="form.fieldsSel"
|
||||
:key="form.datasource + '-' + String(form.orderType)"
|
||||
:options="fieldOptionsDynamic"
|
||||
:props="{ multiple: true, checkStrictly: false, expandTrigger: 'hover', checkOnClickNode: true, checkOnClickLeaf: true }"
|
||||
:teleported="false"
|
||||
collapse-tags
|
||||
collapse-tags-tooltip
|
||||
placeholder="按场景逐级选择，可多选"
|
||||
@visible-change="onCascaderVisible('create', $event)"
|
||||
@change="onFieldsSelChange('create')"
|
||||
/>
|
||||
</div>
|
||||
</el-form-item>
|
||||
|
||||
|
||||
<el-row :gutter="8">
|
||||
<el-col :span="12">
|
||||
<el-form-item label="输出格式" required show-message prop="file_format">
|
||||
<el-select v-model="form.file_format" :teleported="false" placeholder="请选择" style="width:160px">
|
||||
<el-option v-for="opt in formatOptions" :key="opt.value" :label="opt.label" :value="opt.value" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :span="12">
|
||||
<el-form-item label="可见性" required show-message prop="visibility">
|
||||
<el-select v-model="form.visibility" clearable :teleported="false" style="width:160px" placeholder="请选择">
|
||||
<el-option v-for="opt in visibilityOptions" :key="opt.value" :label="opt.label" :value="opt.value" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
</el-row>
|
||||
|
||||
</el-form>
|
||||
<template #footer>
|
||||
<el-button @click="resizeDialog('create', -100)">缩小</el-button>
|
||||
<el-button @click="resizeDialog('create', 100)">放大</el-button>
|
||||
<el-button @click="createVisible=false">取消</el-button>
|
||||
<el-button type="primary" @click="createTemplate">创建</el-button>
|
||||
</template>
|
||||
</el-dialog>
|
||||
<el-dialog v-model="editVisible" title="编辑模板" :width="editWidth">
|
||||
<el-form ref="editFormRef" :model="edit" :rules="editRules" label-width="110px">
|
||||
<el-form-item label="模板名称" required show-message prop="name"><el-input v-model="edit.name" /></el-input></el-form-item>
|
||||
<el-form-item label="数据源">
|
||||
<el-select v-model="edit.datasource" placeholder="选择" :teleported="false" style="width:160px">
|
||||
<el-option v-for="opt in datasourceOptions" :key="opt.value" :label="opt.label" :value="opt.value" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
<el-form-item label="导出场景">
|
||||
<el-select v-model="edit.main_table" placeholder="订单数据" style="width:160px">
|
||||
<el-option v-for="opt in editSceneOptions" :key="opt.value" :label="opt.label" :value="opt.value" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
<el-form-item label="订单类型" required show-message prop="orderType">
|
||||
<el-radio-group v-model="edit.orderType">
|
||||
<el-radio v-for="opt in orderTypeOptionsFor(edit.datasource)" :key="opt.value" :label="opt.value">{{ opt.label }}</el-radio>
|
||||
</el-radio-group>
|
||||
</el-form-item>
|
||||
<el-form-item label="字段选择" required show-message prop="fieldsSel">
|
||||
<div ref="editCascaderRoot">
|
||||
<el-cascader
|
||||
ref="editFieldsCascader"
|
||||
v-model="edit.fieldsSel"
|
||||
:key="edit.datasource + '-' + String(edit.orderType)"
|
||||
:options="editFieldOptionsDynamic"
|
||||
:props="{ multiple: true, checkStrictly: false, expandTrigger: 'hover', checkOnClickNode: true, checkOnClickLeaf: true }"
|
||||
:teleported="false"
|
||||
collapse-tags
|
||||
collapse-tags-tooltip
|
||||
placeholder="按场景逐级选择，可多选"
|
||||
@visible-change="onCascaderVisible('edit', $event)"
|
||||
@change="onFieldsSelChange('edit')"
|
||||
/>
|
||||
</div>
|
||||
</el-form-item>
|
||||
<el-row :gutter="8">
|
||||
<el-col :span="12">
|
||||
<el-form-item label="输出格式">
|
||||
<el-select v-model="edit.file_format" :teleported="false" placeholder="请选择" style="width:160px">
|
||||
<el-option v-for="opt in formatOptions" :key="opt.value" :label="opt.label" :value="opt.value" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :span="12">
|
||||
<el-form-item label="可见性">
|
||||
<el-select v-model="edit.visibility" clearable :disabled="hasUserId" :teleported="false" style="width:160px" placeholder="请选择">
|
||||
<el-option v-for="opt in visibilityOptions" :key="opt.value" :label="opt.label" :value="opt.value" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</el-form>
|
||||
<template #footer>
|
||||
<el-button @click="resizeDialog('edit', -100)">缩小</el-button>
|
||||
<el-button @click="resizeDialog('edit', 100)">放大</el-button>
|
||||
<el-button @click="editVisible=false">取消</el-button>
|
||||
<el-button type="primary" @click="saveEdit">保存</el-button>
|
||||
</template>
|
||||
</el-dialog>
|
||||
<el-dialog v-model="exportVisible" :title="exportTitle" width="1100px">
|
||||
<el-form ref="exportFormRef" :model="exportForm" :rules="exportRules" label-width="110px" status-icon>
|
||||
<div class="section-title">筛选条件</div>
|
||||
<el-row :gutter="8">
|
||||
<el-col :span="24">
|
||||
<el-form-item label="时间范围" required show-message prop="dateRange">
|
||||
<el-date-picker v-model="exportForm.dateRange" type="datetimerange" start-placeholder="开始时间" end-placeholder="结束时间" value-format="YYYY-MM-DD HH:mm:ss" style="width:100%" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
</el-row>
|
||||
<el-row :gutter="8" v-if="isOrder && exportForm.datasource==='ymt'">
|
||||
<el-col :span="8">
|
||||
<el-form-item label="创建者" prop="ymtCreatorId">
|
||||
<el-select v-model.number="exportForm.ymtCreatorId" :disabled="hasUserId" clearable filterable :teleported="false" placeholder="请选择创建者" style="width:100%">
|
||||
<el-option v-for="opt in ymtCreatorOptions" :key="opt.value" :label="opt.label" :value="opt.value" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :span="8">
|
||||
<el-form-item label="客户" prop="ymtMerchantId">
|
||||
<el-select v-model.number="exportForm.ymtMerchantId" :disabled="!exportForm.ymtCreatorId" clearable filterable :teleported="false" placeholder="请选择客户" style="width:100%">
|
||||
<el-option v-for="opt in ymtMerchantOptions" :key="opt.value" :label="opt.label" :value="opt.value" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :span="8">
|
||||
<el-form-item label="活动" prop="ymtActivityId">
|
||||
<el-select v-model.number="exportForm.ymtActivityId" :disabled="!exportForm.ymtMerchantId" clearable filterable :teleported="false" placeholder="请选择活动" style="width:100%">
|
||||
<el-option v-for="opt in ymtActivityOptions" :key="opt.value" :label="opt.label" :value="opt.value" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
</el-row>
|
||||
<el-row :gutter="8" v-if="isOrder && exportForm.datasource==='marketing'">
|
||||
<el-col :span="12">
|
||||
<el-form-item label="创建者" prop="creator">
|
||||
<el-select v-model="exportForm.creatorIds" multiple filterable :disabled="hasUserId" :teleported="false" placeholder="请选择创建者" style="width:100%">
|
||||
<el-option v-for="opt in creatorOptions" :key="opt.value" :label="opt.label" :value="opt.value" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :span="12">
|
||||
<el-form-item label="分销商" prop="resellerId">
|
||||
<el-select v-model.number="exportForm.resellerId" filterable :teleported="false" placeholder="请选择分销商" style="width:100%" :disabled="!hasCreators">
|
||||
<el-option v-for="opt in resellerOptions" :key="opt.value" :label="opt.label||String(opt.value)" :value="opt.value" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
</el-row>
|
||||
<el-row :gutter="8" v-if="isOrder && exportForm.datasource==='marketing'">
|
||||
<el-col :span="12">
|
||||
<el-form-item label="计划" prop="planId">
|
||||
<el-select v-model="exportForm.planId" :disabled="!hasReseller" :teleported="false" filterable placeholder="选择计划">
|
||||
<el-option v-for="opt in planOptions" :key="opt.value" :label="opt.label" :value="opt.value" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
</el-row>
|
||||
|
||||
|
||||
<el-row :gutter="8" v-if="isOrder">
|
||||
<el-col :span="12" v-if="exportType===2">
|
||||
<el-form-item label="立减金批次号" prop="voucherChannelActivityId"><el-input v-model="exportForm.voucherChannelActivityId" placeholder="请输入立减金批次号" /></el-form-item>
|
||||
</el-col>
|
||||
</el-row>
|
||||
|
||||
|
||||
<!-- 输出格式按模板设置，不在此显示 -->
|
||||
</el-form>
|
||||
<template #footer>
|
||||
<el-button @click="exportVisible=false">取消</el-button>
|
||||
<el-button type="primary" :loading="exportSubmitting" :disabled="exportSubmitting" @click="submitExport">{{ exportSubmitting ? '估算中...' : '执行并分析' }}</el-button>
|
||||
</template>
|
||||
</el-dialog>
|
||||
</div>
|
||||
<script src="./vendor/vue.global.prod.js"></script>
|
||||
<script src="./vendor/element-plus.full.min.js"></script>
|
||||
<script src="./main.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,2 @@
|
|||
body{font-family:system-ui,Arial}
|
||||
.section-title{font-weight:600;margin:8px 0}
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Loading…
Reference in New Issue