MapReduce应用程序主REST API允许用户获取正在运行的MapReduce应用程序主控的状态。当前,这等效于正在运行的MapReduce作业。该信息包括应用程序主机正在运行的作业以及所有作业详细信息,例如任务,计数器,配置,尝试等。应通过代理访问应用程序主机。可将该代理配置为在资源管理器或单独的主机上运行。代理URL通常如下所示:http:// proxy-http-address:port / proxy / appid。
MapReduce应用程序主信息资源提供有关该Mapreduce应用程序主信息的总体信息。这包括应用程序ID,启动时间,用户,名称等。
当您请求mapreduce应用程序主信息时,该信息将作为信息对象返回。
项目 | 数据类型 | 描述 |
---|---|---|
appId | 长 | 申请编号 |
startsOn | 长 | 应用程序启动的时间(从纪元开始以毫秒为单位) |
名称 | 串 | 应用程序名称 |
用户 | 串 | 启动应用程序的用户的用户名 |
经过时间 | 长 | 自应用程序启动以来的时间(以毫秒为单位) |
JSON回应
HTTP请求:
获取http:// proxy-http-address:port / proxy / application_1326232085508_0003 / ws / v1 / mapreduce / info
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{ “信息”:{ “ appId”:“ application_1326232085508_0003”, “ startedOn”:1326238244047, “ user”:“ user1”, “ name”:“睡眠工作”, “ elapsedTime”:32374 } }
XML回应
HTTP请求:
接受:application / xml 获取http:// proxy-http-address:port / proxy / application_1326232085508_0003 / ws / v1 / mapreduce / info
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:223 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <信息> <appId> application_1326232085508_0003 </ appId> <name>睡眠作业</ name> <user> user1 </ user> <startedOn> 1326238244047 </ startedOn> <elapsedTime> 32407 </ elapsedTime> </ info>
作业资源提供了在此应用程序主机上运行的作业的列表。另请参见Job API,以获取作业对象的语法。
当您请求作业列表时,该信息将作为作业对象的集合返回。另请参见Job API,以获取作业对象的语法。
项目 | 数据类型 | 描述 |
---|---|---|
工作 | 作业对象数组(JSON)/零个或多个作业对象(XML) | 作业对象的集合 |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{ “职位” : { “工作”:[ { “ runningReduceAttempts”:1, “ reduceProgress”:100, “ failedReduceAttempts”:0, “ newMapAttempts”:0, “ mapsRunning”:0, “ state”:“ RUNNING”, “ successfulReduceAttempts”:0, “ reducesRunning”:1, “ acls”:[ { “ value”:“”, “名称”:“ mapreduce.job.acl-modify-job” }, { “ value”:“”, “名称”:“ mapreduce.job.acl-view-job” } ], “ reducesPending”:0, “ user”:“ user1”, “ reducesTotal”:1 “ mapsCompleted”:1 “ startTime”:1326238769379, “ id”:“ job_1326232085508_4_4”, “ successfulMapAttempts”:1 “ runningMapAttempts”:0, “ newReduceAttempts”:0, “ name”:“睡眠工作”, “ mapsPending”:0, “ elapsedTime”:59377, “ reducesCompleted”:0, “ mapProgress”:100, “诊断”:“”, “ failedMapAttempts”:0, “ killedReduceAttempts”:0, “ mapsTotal”:1 “ uberized”:错误, “ killedMapAttempts”:0, “ finishTime”:0 } ] } }
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:1214 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <职位> <工作> <startTime> 1326238769379 </ startTime> <finishTime> 0 </ finishTime> <elapsedTime> 59416 </ elapsedTime> <id> job_1326232085508_4_4 </ id> <name>睡眠作业</ name> <user> user1 </ user> <状态>运行中</状态> <mapsTotal> 1 </ mapsTotal> <mapsCompleted> 1 </ mapsCompleted> <reducesTotal> 1 </ reducesTotal> <reducesCompleted> 0 </ reducesCompleted> <mapProgress> 100.0 </ mapProgress> <reduceProgress> 100.0 </ reduceProgress> <mapsPending> 0 </ mapsPending> <mapsRunning> 0 </ mapsRunning> <reducesPending> 0 </ reducesPending> <reducesRunning> 1 </ reducesRunning> <uberized> false </ uberized> <diagnostics /> <newReduceAttempts> 0 </ newReduceAttempts> <runningReduceAttempts> 1 </ runningReduceAttempts> <failedReduceAttempts> 0 </ failedReduceAttempts> <killedReduceAttempts> 0 </ killedReduceAttempts> <successfulReduceAttempts> 0 </ successfulReduceAttempts> <newMapAttempts> 0 </ newMapAttempts> <runningMapAttempts> 0 </ runningMapAttempts> <failedMapAttempts> 0 </ failedMapAttempts> <killedMapAttempts> 0 </ killedMapAttempts> <successfulMapAttempts> 1 </ successfulMapAttempts> <acls> <name> mapreduce.job.acl-modify-job </ name> <value> </ value> </ acls> <acls> <name> mapreduce.job.acl-view-job </ name> <value> </ value> </ acls> </ job> </ jobs>
作业资源包含有关由该应用程序主机启动的特定作业的信息。某些字段仅在用户具有权限时才可访问-取决于acl设置。
项目 | 数据类型 | 描述 |
---|---|---|
ID | 串 | 工作编号 |
名称 | 串 | 工作名称 |
用户 | 串 | 用户名 |
州 | 串 | 作业状态-有效值为:NEW,INITED,RUNNING,Succeeded,FAILED,KILL_WAIT,KILLED,ERROR |
开始时间 | 长 | 作业开始的时间(从纪元开始以毫秒为单位) |
finishTime | 长 | 作业完成的时间(从开始到现在的毫秒数) |
经过时间 | 长 | 自作业开始以来经过的时间(毫秒) |
mapsTotal | 整型 | 地图总数 |
mapsCompleted | 整型 | 完成的地图数 |
reduceTotal | 整型 | 减少总数 |
reduceCompleted | 整型 | 完成数量减少 |
诊断 | 串 | 诊断消息 |
超级 | 布尔值 | 指示该作业是否为超级作业-完全在应用程序主服务器中运行 |
mapsPending | 整型 | 仍要运行的地图数量 |
mapsRunning | 整型 | 正在运行的地图数 |
reducePending | 整型 | 减少数量仍需运行 |
reduceRunning | 整型 | 运行次数减少 |
newReduce尝试 | 整型 | 新的减少尝试次数 |
runningReduceAttempts | 整型 | 运行减少尝试的次数 |
failedReduce尝试 | 整型 | 减少尝试失败的次数 |
减少尝试 | 整型 | 减少尝试的失败次数 |
成功减少尝试 | 整型 | 成功减少尝试的次数 |
newMapAttempts | 整型 | 新地图尝试次数 |
runningMapAttempts | 整型 | 正在运行的地图尝试次数 |
failedMapAttempts | 整型 | 失败的映射尝试次数 |
KilledMapAttempts | 整型 | 被杀死的地图尝试次数 |
successMapAttempts | 整型 | 成功的地图尝试次数 |
ACL | acls(json)/零个或多个acls对象(xml)的数组 | ACLS对象的集合 |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 服务器:码头(6.1.26) 内容长度:720
响应主体:
{ “工作”:{ “ runningReduceAttempts”:1, “ reduceProgress”:100, “ failedReduceAttempts”:0, “ newMapAttempts”:0, “ mapsRunning”:0, “ state”:“ RUNNING”, “ successfulReduceAttempts”:0, “ reducesRunning”:1, “ acls”:[ { “ value”:“”, “名称”:“ mapreduce.job.acl-modify-job” }, { “ value”:“”, “名称”:“ mapreduce.job.acl-view-job” } ], “ reducesPending”:0, “ user”:“ user1”, “ reducesTotal”:1 “ mapsCompleted”:1 “ startTime”:1326238769379, “ id”:“ job_1326232085508_4_4”, “ successfulMapAttempts”:1 “ runningMapAttempts”:0, “ newReduceAttempts”:0, “ name”:“睡眠工作”, “ mapsPending”:0, “ elapsedTime”:59437, “ reducesCompleted”:0, “ mapProgress”:100, “诊断”:“”, “ failedMapAttempts”:0, “ killedReduceAttempts”:0, “ mapsTotal”:1 “ uberized”:错误, “ killedMapAttempts”:0, “ finishTime”:0 } }
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:1201 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <工作> <startTime> 1326238769379 </ startTime> <finishTime> 0 </ finishTime> <elapsedTime> 59474 </ elapsedTime> <id> job_1326232085508_4_4 </ id> <name>睡眠作业</ name> <user> user1 </ user> <状态>运行中</状态> <mapsTotal> 1 </ mapsTotal> <mapsCompleted> 1 </ mapsCompleted> <reducesTotal> 1 </ reducesTotal> <reducesCompleted> 0 </ reducesCompleted> <mapProgress> 100.0 </ mapProgress> <reduceProgress> 100.0 </ reduceProgress> <mapsPending> 0 </ mapsPending> <mapsRunning> 0 </ mapsRunning> <reducesPending> 0 </ reducesPending> <reducesRunning> 1 </ reducesRunning> <uberized> false </ uberized> <diagnostics /> <newReduceAttempts> 0 </ newReduceAttempts> <runningReduceAttempts> 1 </ runningReduceAttempts> <failedReduceAttempts> 0 </ failedReduceAttempts> <killedReduceAttempts> 0 </ killedReduceAttempts> <successfulReduceAttempts> 0 </ successfulReduceAttempts> <newMapAttempts> 0 </ newMapAttempts> <runningMapAttempts> 0 </ runningMapAttempts> <failedMapAttempts> 0 </ failedMapAttempts> <killedMapAttempts> 0 </ killedMapAttempts> <successfulMapAttempts> 1 </ successfulMapAttempts> <acls> <name> mapreduce.job.acl-modify-job </ name> <value> </ value> </ acls> <acls> <name> mapreduce.job.acl-view-job </ name> <value> </ value> </ acls> </ job>
使用作业尝试API,您可以获得代表作业尝试的资源集合。在此资源上运行GET操作时,您将获得一个Job Attempt对象的集合。
当您请求作业尝试列表时,该信息将作为作业尝试对象数组返回。
项目 | 数据类型 | 描述 |
---|---|---|
工作尝试 | 作业尝试对象数组(JSON)/零个或多个作业尝试对象(XML) | 作业尝试对象的集合 |
项目 | 数据类型 | 描述 |
---|---|---|
ID | 串 | 工作尝试编号 |
nodeId | 串 | 尝试运行的节点的节点ID |
nodeHttpAddress | 串 | 尝试运行的节点的节点http地址 |
logsLink | 串 | 作业尝试日志的http链接 |
containerId | 串 | 尝试作业的容器的ID |
开始时间 | 长 | 尝试的开始时间(自纪元以来以毫秒为单位) |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / jobattempts
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{ “ jobAttempts”:{ “ jobAttempt”:[ { “ nodeId”:“ host.domain.com:8041”, “ nodeHttpAddress”:“ host.domain.com:8042”, “ startTime”:1326238773493, “ id”:1 “ logsLink”:“ http://host.domain.com:8042/node/containerlogs/container_1326232085508_0004_01_000001”, “ containerId”:“ container_1326232085508_0004_01_000001” } ] } }
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / jobattempts 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:498 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <jobAttempts> <jobAttempt> <nodeHttpAddress> host.domain.com:8042 </ nodeHttpAddress> <nodeId> host.domain.com:8041 </ nodeId> <id> 1 </ id> <startTime> 1326238773493 </ startTime> <containerId> container_1326232085508_0004_01_000001 </ containerId> <logsLink> http://host.domain.com:8042/node/containerlogs/container_1326232085508_0004_01_000001 </ logsLink> </ jobAttempt> </ jobAttempts>
使用作业计数器API,您可以反对代表该作业的所有计数器的资源集合。
项目 | 数据类型 | 描述 |
---|---|---|
ID | 串 | 工作编号 |
柜台集团 | counterGroup对象的数组(JSON)/零个或多个counterGroup对象的XML | 计数器组对象的集合 |
项目 | 数据类型 | 描述 |
---|---|---|
名称 | 串 | 柜台名称 |
reduceCounterValue | 长 | 减少任务的计数器值 |
mapCounterValue | 长 | 地图任务的对价 |
totalCounterValue | 长 | 所有任务的对价 |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / counters
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{ “ jobCounters”:{ “ id”:“ job_1326232085508_4_4”, “ counterGroup”:[ { “ counterGroupName”:“随机播放错误”, “柜台”:[ { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ BAD_ID” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“连接” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ IO_ERROR” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ WRONG_LENGTH” },{ “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ WRONG_MAP” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ WRONG_REDUCE” } ] }, { “ counterGroupName”:“ org.apache.hadoop.mapreduce.FileSystemCounter”, “柜台”:[ { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:2483, “名称”:“ FILE_BYTES_READ” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:108763, “名称”:“ FILE_BYTES_WRITTEN” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ FILE_READ_OPS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ FILE_LARGE_READ_OPS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ FILE_WRITE_OPS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:48, “名称”:“ HDFS_BYTES_READ” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ HDFS_BYTES_WRITTEN” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:1 “名称”:“ HDFS_READ_OPS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ HDFS_LARGE_READ_OPS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ HDFS_WRITE_OPS” } ] }, { “ counterGroupName”:“ org.apache.hadoop.mapreduce.TaskCounter”, “柜台”:[ { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:1 “名称”:“ MAP_INPUT_RECORDS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:1200, “名称”:“ MAP_OUTPUT_RECORDS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:4800, “名称”:“ MAP_OUTPUT_BYTES” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:2235, “名称”:“ MAP_OUTPUT_MATERIALIZED_BYTES” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:48, “名称”:“ SPLIT_RAW_BYTES” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ COMBINE_INPUT_RECORDS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ COMBINE_OUTPUT_RECORDS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:460, “名称”:“ REDUCE_INPUT_GROUPS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:2235, “名称”:“ REDUCE_SHUFFLE_BYTES” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:460, “名称”:“ REDUCE_INPUT_RECORDS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ REDUCE_OUTPUT_RECORDS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:1200, “名称”:“ SPILLED_RECORDS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:1 “名称”:“ SHUFFLED_MAPS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ FAILED_SHUFFLE” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:1 “名称”:“ MERGED_MAP_OUTPUTS” },{ “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:58 “名称”:“ GC_TIME_MILLIS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:1580, “名称”:“ CPU_MILLISECONDS” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:462643200, “名称”:“ PHYSICAL_MEMORY_BYTES” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:2149728256, “名称”:“ VIRTUAL_MEMORY_BYTES” }, { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:357957632, “名称”:“ COMMITTED_HEAP_BYTES” } ] }, { “ counterGroupName”:“ org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter”, “柜台”:[ { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ BYTES_READ” } ] }, { “ counterGroupName”:“ org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter”, “柜台”:[ { “ reduceCounterValue”:0, “ mapCounterValue”:0, “ totalCounterValue”:0, “名称”:“ BYTES_WRITTEN” } ] } ] } }
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / counters 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:7027 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <jobCounters> <id> job_1326232085508_4_4 </ id> <counterGroup> <counterGroupName>随机播放错误</ counterGroupName> <计数器> <name> BAD_ID </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> CONNECTION </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> IO_ERROR </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> WRONG_LENGTH </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> WRONG_MAP </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> WRONG_REDUCE </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> </ counterGroup> <counterGroup> <counterGroupName> org.apache.hadoop.mapreduce.FileSystemCounter </ counterGroupName> <计数器> <name> FILE_BYTES_READ </ name> <totalCounterValue> 2483 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> FILE_BYTES_WRITTEN </ name> <totalCounterValue> 108763 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <名称> FILE_READ_OPS </名称> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <名称> FILE_LARGE_READ_OPS </名称> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <名称> FILE_WRITE_OPS </名称> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> HDFS_BYTES_READ </ name> <totalCounterValue> 48 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> HDFS_BYTES_WRITTEN </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> HDFS_READ_OPS </ name> <totalCounterValue> 1 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> HDFS_LARGE_READ_OPS </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> HDFS_WRITE_OPS </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> </ counterGroup> <counterGroup> <counterGroupName> org.apache.hadoop.mapreduce.TaskCounter </ counterGroupName> <计数器> <名称> MAP_INPUT_RECORDS </名称> <totalCounterValue> 1 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <名称> MAP_OUTPUT_RECORDS </名称> <totalCounterValue> 1200 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <名称> MAP_OUTPUT_BYTES </名称> <totalCounterValue> 4800 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> MAP_OUTPUT_MATERIALIZED_BYTES </ name> <totalCounterValue> 2235 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> SPLIT_RAW_BYTES </ name> <totalCounterValue> 48 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> COMBINE_INPUT_RECORDS </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> COMBINE_OUTPUT_RECORDS </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> REDUCE_INPUT_GROUPS </ name> <totalCounterValue> 460 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> REDUCE_SHUFFLE_BYTES </ name> <totalCounterValue> 2235 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> REDUCE_INPUT_RECORDS </ name> <totalCounterValue> 460 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> REDUCE_OUTPUT_RECORDS </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> SPILLED_RECORDS </ name> <totalCounterValue> 1200 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> SHUFFLED_MAPS </ name> <totalCounterValue> 1 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> FAILED_SHUFFLE </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> MERGED_MAP_OUTPUTS </ name> <totalCounterValue> 1 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> GC_TIME_MILLIS </ name> <totalCounterValue> 58 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <名称> CPU_MILLISECONDS </名称> <totalCounterValue> 1580 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> PHYSICAL_MEMORY_BYTES </ name> <totalCounterValue> 462643200 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> VIRTUAL_MEMORY_BYTES </ name> <totalCounterValue> 2149728256 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> <计数器> <name> COMMITTED_HEAP_BYTES </ name> <totalCounterValue> 357957632 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> </ counterGroup> <counterGroup> <counterGroupName> org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter </ counterGroupName> <计数器> <name> BYTES_READ </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> </ counterGroup> <counterGroup> <counterGroupName> org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter </ counterGroupName> <counter> <name> BYTES_WRITTEN </ name> <totalCounterValue> 0 </ totalCounterValue> <mapCounterValue> 0 </ mapCounterValue> <reduceCounterValue> 0 </ reduceCounterValue> </ counter> </ counterGroup> </ jobCounters>
作业配置资源包含有关此作业的作业配置的信息。
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / conf
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
如果输出很大,这是输出的一小段。实际输出包含作业配置文件中的每个属性。
{ “ conf”:{ “ path”:“ hdfs://host.domain.com:9000 / user / user1 / .staging / job_1326232085508_0004 / job.xml”, “财产”:[ { “ value”:“ / home / hadoop / hdfs / data”, “ name”:“ dfs.datanode.data.dir”, “源”:[“ hdfs-site.xml”,“ job.xml”] }, { “ value”:“ org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer”, “ name”:“ hadoop.http.filter.initializers” “源”:[“以编程方式”,“ job.xml”] }, { “ value”:“ / home / hadoop / tmp”, “名称”:“ mapreduce.cluster.temp.dir” “源”:[“ mapred-site.xml”] }, ... ] } }
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / conf 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:552 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <conf> <path> hdfs://host.domain.com:9000 / user / user1 / .staging / job_1326232085508_0004 / job.xml </ path> <属性> <name> dfs.datanode.data.dir </ name> <value> / home / hadoop / hdfs / data </ value> <source> hdfs-site.xml </ source> <source> job.xml </ source> </ property> <属性> <name> hadoop.http.filter.initializers </ name> <value> org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer </ value> <source>以编程方式</ source> <source> job.xml </ source> </ property> <属性> <name> mapreduce.cluster.temp.dir </ name> <值> / home / hadoop / tmp </值> <source> mapred-site.xml </ source> </ property> ... </ conf>
使用任务API,您可以获得代表作业的所有任务的资源集合。在此资源上运行GET操作时,您将获得任务对象的集合。
当您请求任务列表时,信息将作为任务对象数组返回。另请参阅任务API,以获取任务对象的语法。
项目 | 数据类型 | 描述 |
---|---|---|
任务 | 任务对象数组(JSON)/零个或多个任务对象(XML) | 任务对象的集合 |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{ “任务” : { “任务”:[ { “进度”:100, “ elapsedTime”:2768, “ state”:“ Succeeded”, “ startTime”:1326238773493, “ id”:“ task_1326232085508_4_4_m_0”, “ type”:“ MAP”, “ successfulAttempt”:“ attempt_1326232085508_4_4_m_0_0”, “ finishTime”:1326238776261 }, { “进度”:100, “ elapsedTime”:0, “ state”:“ RUNNING”, “ startTime”:1326238777460, “ id”:“ task_1326232085508_4_4_r_0”, “ type”:“ REDUCE”, “ successfulAttempt”:“”, “ finishTime”:0 } ] } }
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:603 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <任务> <任务> <startTime> 1326238773493 </ startTime> <finishTime> 1326238776261 </ finishTime> <elapsedTime> 2768 </ elapsedTime> <progress> 100.0 </ progress> <id> task_1326232085508_4_4_m_0 </ id> <state>成功</ state> <type> MAP </ type> <successfulAttempt> attempt_1326232085508_4_4_m_0_0 </ successfulAttempt> </ task> <任务> <startTime> 1326238777460 </ startTime> <finishTime> 0 </ finishTime> <elapsedTime> 0 </ elapsedTime> <progress> 100.0 </ progress> <id> task_1326232085508_4_4_r_0 </ id> <状态>运行中</状态> <type> REDUCE </ type> <successfulAttempt /> </ task> </ tasks>
任务资源包含有关作业中特定任务的信息。
项目 | 数据类型 | 描述 |
---|---|---|
ID | 串 | 任务ID |
州 | 串 | 任务的状态-有效值为:NEW,SCHEDULED,RUNNING,Succeeded,FAILED,KILL_WAIT,KILLED |
类型 | 串 | 任务类型-MAP或REDUCE |
成功尝试 | 串 | 上次成功尝试的ID |
进展 | 浮动 | 任务进度百分比 |
开始时间 | 长 | 任务开始的时间(从纪元开始以毫秒为单位) |
finishTime | 长 | 任务完成的时间(从纪元开始以毫秒为单位) |
经过时间 | 长 | 自应用程序启动以来经过的时间(以毫秒为单位) |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{ “任务”:{ “进度”:100, “ elapsedTime”:0, “ state”:“ RUNNING”, “ startTime”:1326238777460, “ id”:“ task_1326232085508_4_4_r_0”, “ type”:“ REDUCE”, “ successfulAttempt”:“”, “ finishTime”:0 } }
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:299 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <任务> <startTime> 1326238777460 </ startTime> <finishTime> 0 </ finishTime> <elapsedTime> 0 </ elapsedTime> <progress> 100.0 </ progress> <id> task_1326232085508_4_4_r_0 </ id> <状态>运行中</状态> <type> REDUCE </ type> <successfulAttempt /> </ task>
使用任务计数器API,您可以反对代表该任务所有计数器的资源集合。
项目 | 数据类型 | 描述 |
---|---|---|
ID | 串 | 任务ID |
taskcounterGroup | counterGroup对象的数组(JSON)/零个或多个counterGroup对象的XML | 计数器组对象的集合 |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 / counters
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{ “ jobTaskCounters”:{ “ id”:“ task_1326232085508_4_4_r_0”, “ taskCounterGroup”:[ { “ counterGroupName”:“ org.apache.hadoop.mapreduce.FileSystemCounter”, “柜台”:[ { “值”:2363, “名称”:“ FILE_BYTES_READ” }, { “值”:54372, “名称”:“ FILE_BYTES_WRITTEN” }, { “值”:0, “名称”:“ FILE_READ_OPS” }, { “值”:0, “名称”:“ FILE_LARGE_READ_OPS” }, { “值”:0, “名称”:“ FILE_WRITE_OPS” }, { “值”:0, “名称”:“ HDFS_BYTES_READ” }, { “值”:0, “名称”:“ HDFS_BYTES_WRITTEN” }, { “值”:0, “名称”:“ HDFS_READ_OPS” }, { “值”:0, “名称”:“ HDFS_LARGE_READ_OPS” }, { “值”:0, “名称”:“ HDFS_WRITE_OPS” } ] }, { “ counterGroupName”:“ org.apache.hadoop.mapreduce.TaskCounter”, “柜台”:[ { “值”:0, “名称”:“ COMBINE_INPUT_RECORDS” }, { “值”:0, “名称”:“ COMBINE_OUTPUT_RECORDS” }, { “值”:460, “名称”:“ REDUCE_INPUT_GROUPS” }, { “值”:2235, “名称”:“ REDUCE_SHUFFLE_BYTES” }, { “值”:460, “名称”:“ REDUCE_INPUT_RECORDS” }, { “值”:0, “名称”:“ REDUCE_OUTPUT_RECORDS” }, { “值”:0, “名称”:“ SPILLED_RECORDS” }, { “值”:1 “名称”:“ SHUFFLED_MAPS” }, { “值”:0, “名称”:“ FAILED_SHUFFLE” }, { “值”:1 “名称”:“ MERGED_MAP_OUTPUTS” }, { “值”:26, “名称”:“ GC_TIME_MILLIS” }, { “值”:860, “名称”:“ CPU_MILLISECONDS” }, { “值”:107839488, “名称”:“ PHYSICAL_MEMORY_BYTES” }, { “值”:1123147776, “名称”:“ VIRTUAL_MEMORY_BYTES” }, { “值”:57475072, “名称”:“ COMMITTED_HEAP_BYTES” } ] }, { “ counterGroupName”:“随机播放错误”, “柜台”:[ { “值”:0, “名称”:“ BAD_ID” }, { “值”:0, “名称”:“连接” }, { “值”:0, “名称”:“ IO_ERROR” }, { “值”:0, “名称”:“ WRONG_LENGTH” }, { “值”:0, “名称”:“ WRONG_MAP” }, { “值”:0, “名称”:“ WRONG_REDUCE” } ] }, { “ counterGroupName”:“ org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter”, “柜台”:[ { “值”:0, “名称”:“ BYTES_WRITTEN” } ] } ] } }
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 / counters 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:2660 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <jobTaskCounters> <id> task_1326232085508_4_4_r_0 </ id> <taskCounterGroup> <counterGroupName> org.apache.hadoop.mapreduce.FileSystemCounter </ counterGroupName> <计数器> <name> FILE_BYTES_READ </ name> <value> 2363 </ value> </ counter> <计数器> <name> FILE_BYTES_WRITTEN </ name> <value> 54372 </ value> </ counter> <计数器> <名称> FILE_READ_OPS </名称> <value> 0 </ value> </ counter> <计数器> <名称> FILE_LARGE_READ_OPS </名称> <value> 0 </ value> </ counter> <计数器> <名称> FILE_WRITE_OPS </名称> <value> 0 </ value> </ counter> <计数器> <name> HDFS_BYTES_READ </ name> <value> 0 </ value> </ counter> <计数器> <name> HDFS_BYTES_WRITTEN </ name> <value> 0 </ value> </ counter> <计数器> <name> HDFS_READ_OPS </ name> <value> 0 </ value> </ counter> <计数器> <name> HDFS_LARGE_READ_OPS </ name> <value> 0 </ value> </ counter> <计数器> <name> HDFS_WRITE_OPS </ name> <value> 0 </ value> </ counter> </ taskCounterGroup> <taskCounterGroup> <counterGroupName> org.apache.hadoop.mapreduce.TaskCounter </ counterGroupName> <计数器> <name> COMBINE_INPUT_RECORDS </ name> <value> 0 </ value> </ counter> <计数器> <name> COMBINE_OUTPUT_RECORDS </ name> <value> 0 </ value> </ counter> <计数器> <name> REDUCE_INPUT_GROUPS </ name> <value> 460 </ value> </ counter> <计数器> <name> REDUCE_SHUFFLE_BYTES </ name> <value> 2235 </ value> </ counter> <计数器> <name> REDUCE_INPUT_RECORDS </ name> <value> 460 </ value> </ counter> <计数器> <name> REDUCE_OUTPUT_RECORDS </ name> <value> 0 </ value> </ counter> <计数器> <name> SPILLED_RECORDS </ name> <value> 0 </ value> </ counter> <计数器> <name> SHUFFLED_MAPS </ name> <value> 1 </ value> </ counter> <计数器> <name> FAILED_SHUFFLE </ name> <value> 0 </ value> </ counter> <计数器> <name> MERGED_MAP_OUTPUTS </ name> <value> 1 </ value> </ counter> <计数器> <name> GC_TIME_MILLIS </ name> <value> 26 </ value> </ counter> <计数器> <名称> CPU_MILLISECONDS </名称> <value> 860 </ value> </ counter> <计数器> <name> PHYSICAL_MEMORY_BYTES </ name> <value> 107839488 </ value> </ counter> <计数器> <name> VIRTUAL_MEMORY_BYTES </ name> <value> 1123147776 </ value> </ counter> <计数器> <name> COMMITTED_HEAP_BYTES </ name> <value> 57475072 </ value> </ counter> </ taskCounterGroup> <taskCounterGroup> <counterGroupName>随机播放错误</ counterGroupName> <计数器> <name> BAD_ID </ name> <value> 0 </ value> </ counter> <计数器> <name> CONNECTION </ name> <value> 0 </ value> </ counter> <计数器> <name> IO_ERROR </ name> <value> 0 </ value> </ counter> <计数器> <name> WRONG_LENGTH </ name> <value> 0 </ value> </ counter> <计数器> <name> WRONG_MAP </ name> <value> 0 </ value> </ counter> <计数器> <name> WRONG_REDUCE </ name> <value> 0 </ value> </ counter> </ taskCounterGroup> <taskCounterGroup> <counterGroupName> org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter </ counterGroupName> <计数器> <name> BYTES_WRITTEN </ name> <value> 0 </ value> </ counter> </ taskCounterGroup> </ jobTaskCounters>
使用任务尝试API,您可以获得代表作业中任务尝试的资源集合。在此资源上运行GET操作时,将获得“任务尝试对象”的集合。
当您请求任务尝试列表时,该信息将作为任务尝试对象数组返回。另请参阅Task Attempt API,以获取任务对象的语法。
项目 | 数据类型 | 描述 |
---|---|---|
taskAttempt | 任务尝试对象(JSON)/零个或多个任务尝试对象(XML)的数组 | 任务尝试对象的集合 |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 /尝试
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{ “ taskAttempts”:{ “ taskAttempt”:[ { “ elapsedMergeTime”:47, “ shuffleFinishTime”:1326238780052, “ assignedContainerId”:“ container_1326232085508_0004_01_000003”, “进度”:100, “ elapsedTime”:0, “ state”:“ RUNNING”, “ elapsedShuffleTime”:2592, “ mergeFinishTime”:1326238780099, “ rack”:“ / 98.139.92.0”, “ elapsedReduceTime”:0, “ nodeHttpAddress”:“ host.domain.com:8042”, “ type”:“ REDUCE”, “ startTime”:1326238777460, “ id”:“ attempt_1326232085508_4_4_r_0_0”, “ finishTime”:0 } ] } }
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 /尝试 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:807 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <taskAttempts> <taskAttempt> <startTime> 1326238777460 </ startTime> <finishTime> 0 </ finishTime> <elapsedTime> 0 </ elapsedTime> <progress> 100.0 </ progress> <id> attempt_1326232085508_4_4_r_0_0 </ id> <rack> /98.139.92.0 </ rack> <状态>运行中</状态> <nodeHttpAddress> host.domain.com:8042 </ nodeHttpAddress> <type> REDUCE </ type> <assignedContainerId> container_1326232085508_0004_01_000003 </ assignedContainerId> <shuffleFinishTime> 1326238780052 </ shuffleFinishTime> <mergeFinishTime> 1326238780099 </ mergeFinishTime> <elapsedShuffleTime> 2592 </ elapsedShuffleTime> <elapsedMergeTime> 47 </ elapsedMergeTime> <elapsedReduceTime> 0 </ elapsedReduceTime> </ taskAttempt> </ taskAttempts>
任务尝试资源包含有关作业中特定任务尝试的信息。
项目 | 数据类型 | 描述 |
---|---|---|
ID | 串 | 任务ID |
架 | 串 | 机架 |
州 | 串 | 任务尝试的状态-有效值为:NEW,UNASSIGNED,ASSIGNED,RUNNING,COMMIT_PENDING,SUCCESS_CONTAINER_CLEANUP,SUCCEEDED,FAIL_CONTAINER_CLEANUP,FAIL_TASK_CLEANUP,FAILED,KILL_CONTAINER_CLEANUP,KILL_TAIL_ |
类型 | 串 | 任务类型 |
AssignedContainerId | 串 | 此尝试分配给的容器ID |
nodeHttpAddress | 串 | 尝试执行此任务的节点的http地址 |
诊断 | 串 | 诊断消息 |
进展 | 浮动 | 任务尝试的进度百分比 |
开始时间 | 长 | 任务尝试开始的时间(自时期起以毫秒为单位) |
finishTime | 长 | 任务尝试完成的时间(自纪元以来以毫秒为单位) |
经过时间 | 长 | 自任务开始尝试以来经过的时间(以毫秒为单位) |
对于减少任务尝试,您还具有以下字段:
项目 | 数据类型 | 描述 |
---|---|---|
shuffleFinishTime | 长 | 随机播放结束的时间(自纪元以来以毫秒为单位) |
mergeFinishTime | 长 | 合并完成的时间(自纪元以来以毫秒为单位) |
经过的ShuffleTime | 长 | 随机播放阶段完成所需的时间(减少任务开始和随机播放完成之间的时间,以毫秒为单位) |
经过的合并时间 | 长 | 合并阶段完成所需的时间(混洗完成和合并完成之间的时间,以毫秒为单位) |
经过减少时间 | 长 | 还原阶段完成所需的时间(从合并完成到还原任务结束之间的时间,以毫秒为单位) |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 / attempts / attempt_1326232085508_4_4_r_0_0
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{ “ taskAttempt”:{ “ elapsedMergeTime”:47, “ shuffleFinishTime”:1326238780052, “ assignedContainerId”:“ container_1326232085508_0004_01_000003”, “进度”:100, “ elapsedTime”:0, “ state”:“ RUNNING”, “ elapsedShuffleTime”:2592, “ mergeFinishTime”:1326238780099, “ rack”:“ / 98.139.92.0”, “ elapsedReduceTime”:0, “ nodeHttpAddress”:“ host.domain.com:8042”, “ startTime”:1326238777460, “ id”:“ attempt_1326232085508_4_4_r_0_0”, “ type”:“ REDUCE”, “ finishTime”:0 } }
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 / attempts / attempt_1326232085508_4_4_r_0_0 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:691 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <taskAttempt> <startTime> 1326238777460 </ startTime> <finishTime> 0 </ finishTime> <elapsedTime> 0 </ elapsedTime> <progress> 100.0 </ progress> <id> attempt_1326232085508_4_4_r_0_0 </ id> <rack> /98.139.92.0 </ rack> <状态>运行中</状态> <nodeHttpAddress> host.domain.com:8042 </ nodeHttpAddress> <type> REDUCE </ type> <assignedContainerId> container_1326232085508_0004_01_000003 </ assignedContainerId> <shuffleFinishTime> 1326238780052 </ shuffleFinishTime> <mergeFinishTime> 1326238780099 </ mergeFinishTime> <elapsedShuffleTime> 2592 </ elapsedShuffleTime> <elapsedMergeTime> 47 </ elapsedMergeTime> <elapsedReduceTime> 0 </ elapsedReduceTime> </ taskAttempt>
使用任务尝试状态API,您可以使用状态设置为“ KILLED”的PUT请求修改运行任务尝试的状态,从而查询提交的任务尝试的状态以及终止正在运行的任务尝试。要执行PUT操作,必须为AM Web服务设置身份验证。另外,您必须被授权杀死任务尝试。当前,您只能将状态更改为“已杀死”。尝试将状态更改为其他任何状态都会导致400错误响应。下面是未授权和错误请求错误的示例。当您执行成功的PUT时,初始响应可能是202。您可以通过重复PUT请求直到得到200,使用GET方法查询状态或查询任务尝试信息并检查,来确认该应用已被杀死。状态。在以下示例中,
请注意,为了终止任务尝试,必须为HTTP接口设置身份验证筛选器。该功能要求在HttpServletRequest中设置用户名。如果未设置任何过滤器,则响应将为“未经授权”响应。
此功能目前处于Alpha阶段,将来可能会更改。
当您请求应用程序状态时,返回的信息包含以下字段
项目 | 数据类型 | 描述 |
---|---|---|
州 | 串 | 应用程序状态-可以是“ NEW”,“ STARTING”,“ RUNNING”,“ COMMIT_PENDING”,“ Succeeded”,“ FAILED”,“ KILLED”之一 |
JSON回应
HTTP请求
GET http:// proxy-http-address:port / proxy / application_1429692837321_0001 / ws / v1 / mapreduce / jobs / job_1429692837321_0001 / tasks / task_1429692837321_0001_m_000000 / attempts / attempt_1429692837321_0001_m_000000_0 / state
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 服务器:码头(6.1.26) 内容长度:20
响应主体:
{ “ state”:“ STARTING” }
HTTP请求
PUT http:// proxy-http-address:port / proxy / application_1429692837321_0001 / ws / v1 / mapreduce / jobs / job_1429692837321_0001 / tasks / task_1429692837321_0001_m_000000 / attempts / attempt_1429692837321_0001_m_000000_0 / state
请求正文:
{ “ state”:“ KILLED” }
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 服务器:码头(6.1.26) 内容长度:18
响应主体:
{ “ state”:“ KILLED” }
XML回应
HTTP请求
GET http:// proxy-http-address:port / proxy / application_1429692837321_0001 / ws / v1 / mapreduce / jobs / job_1429692837321_0001 / tasks / task_1429692837321_0001_m_000000 / attempts / attempt_1429692837321_0001_m_000000_0 / state
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 服务器:码头(6.1.26) 内容长度:121
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <jobTaskAttemptState> <状态>开始</状态> </ jobTaskAttemptState>
HTTP请求
PUT http:// proxy-http-address:port / proxy / application_1429692837321_0001 / ws / v1 / mapreduce / jobs / job_1429692837321_0001 / tasks / task_1429692837321_0001_m_000000 / attempts / attempt_1429692837321_0001_m_000000_0 / state
请求正文:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <jobTaskAttemptState> <state> KILLED </ state> </ jobTaskAttemptState>
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 服务器:码头(6.1.26) 内容长度:121
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <jobTaskAttemptState> <state> KILLED </ state> </ jobTaskAttemptState>
未经授权的错误响应
HTTP请求
PUT http:// proxy-http-address:port / proxy / application_1429692837321_0001 / ws / v1 / mapreduce / jobs / job_1429692837321_0001 / tasks / task_1429692837321_0001_m_000000 / attempts / attempt_1429692837321_0001_m_000000_0 / state
请求正文:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <jobTaskAttemptState> <state> KILLED </ state> </ jobTaskAttemptState>
响应标题:
HTTP / 1.1 403未经授权 内容类型:application / json 服务器:码头(6.1.26)
错误的请求错误响应
HTTP请求
PUT http:// proxy-http-address:port / proxy / application_1429692837321_0001 / ws / v1 / mapreduce / jobs / job_1429692837321_0001 / tasks / task_1429692837321_0001_m_000000 / attempts / attempt_1429692837321_0001_m_000000_0 / state
请求正文:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <jobTaskAttemptState> <状态>运行中</状态> </ jobTaskAttemptState>
响应标题:
HTTP / 1.1 400 内容长度:295 内容类型:application / xml 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <RemoteException> <exception> BadRequestException </ exception> <message> java.lang.Exception:仅允许将“ KILLED”作为目标状态。</ message> <javaClassName> org.apache.hadoop.yarn.webapp.BadRequestException </ javaClassName> </ RemoteException>
使用任务尝试计数器API,您可以反对代表该任务尝试的所有计数器的资源集合。
项目 | 数据类型 | 描述 |
---|---|---|
ID | 串 | 任务尝试ID |
taskAttemptcounterGroup | 任务尝试counterGroup对象(JSON)/零个或多个任务尝试counterGroup对象(XML)的数组 | 任务尝试计数器组对象的集合 |
项目 | 数据类型 | 描述 |
---|---|---|
counterGroupName | 串 | 柜台组名称 |
计数器 | 计数器对象数组(JSON)/零个或多个计数器对象(XML) | 柜台对象的集合 |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 / attempts / attempt_1326232085508_4_4_r_0_0 / counters
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{ “ jobTaskAttemptCounters”:{ “ taskAttemptCounterGroup”:[ { “ counterGroupName”:“ org.apache.hadoop.mapreduce.FileSystemCounter”, “柜台”:[ { “值”:2363, “名称”:“ FILE_BYTES_READ” }, { “值”:54372, “名称”:“ FILE_BYTES_WRITTEN” }, { “值”:0, “名称”:“ FILE_READ_OPS” }, { “值”:0, “名称”:“ FILE_LARGE_READ_OPS” }, { “值”:0, “名称”:“ FILE_WRITE_OPS” }, { “值”:0, “名称”:“ HDFS_BYTES_READ” }, { “值”:0, “名称”:“ HDFS_BYTES_WRITTEN” }, { “值”:0, “名称”:“ HDFS_READ_OPS” }, { “值”:0, “名称”:“ HDFS_LARGE_READ_OPS” }, { “值”:0, “名称”:“ HDFS_WRITE_OPS” } ] }, { “ counterGroupName”:“ org.apache.hadoop.mapreduce.TaskCounter”, “柜台”:[ { “值”:0, “名称”:“ COMBINE_INPUT_RECORDS” }, { “值”:0, “名称”:“ COMBINE_OUTPUT_RECORDS” }, { “值”:460, “名称”:“ REDUCE_INPUT_GROUPS” }, { “值”:2235, “名称”:“ REDUCE_SHUFFLE_BYTES” }, { “值”:460, “名称”:“ REDUCE_INPUT_RECORDS” }, { “值”:0, “名称”:“ REDUCE_OUTPUT_RECORDS” }, { “值”:0, “名称”:“ SPILLED_RECORDS” }, { “值”:1 “名称”:“ SHUFFLED_MAPS” }, { “值”:0, “名称”:“ FAILED_SHUFFLE” }, { “值”:1 “名称”:“ MERGED_MAP_OUTPUTS” }, { “值”:26, “名称”:“ GC_TIME_MILLIS” }, { “值”:860, “名称”:“ CPU_MILLISECONDS” }, { “值”:107839488, “名称”:“ PHYSICAL_MEMORY_BYTES” }, { “值”:1123147776, “名称”:“ VIRTUAL_MEMORY_BYTES” }, { “值”:57475072, “名称”:“ COMMITTED_HEAP_BYTES” } ] }, { “ counterGroupName”:“随机播放错误”, “柜台”:[ { “值”:0, “名称”:“ BAD_ID” }, { “值”:0, “名称”:“连接” }, { “值”:0, “名称”:“ IO_ERROR” }, { “值”:0, “名称”:“ WRONG_LENGTH” }, { “值”:0, “名称”:“ WRONG_MAP” }, { “值”:0, “名称”:“ WRONG_REDUCE” } ] }, { “ counterGroupName”:“ org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter”, “柜台”:[ { “值”:0, “名称”:“ BYTES_WRITTEN” } ] } ], “ id”:“ attempt_1326232085508_4_4_r_0_0” } }
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 / attempts / attempt_1326232085508_4_4_r_0_0 / counters 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:2735 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <jobTaskAttemptCounters> <id> attempt_1326232085508_4_4_r_0_0 </ id> <taskAttemptCounterGroup> <counterGroupName> org.apache.hadoop.mapreduce.FileSystemCounter </ counterGroupName> <计数器> <name> FILE_BYTES_READ </ name> <value> 2363 </ value> </ counter> <计数器> <name> FILE_BYTES_WRITTEN </ name> <value> 54372 </ value> </ counter> <计数器> <名称> FILE_READ_OPS </名称> <value> 0 </ value> </ counter> <计数器> <名称> FILE_LARGE_READ_OPS </名称> <value> 0 </ value> </ counter> <计数器> <名称> FILE_WRITE_OPS </名称> <value> 0 </ value> </ counter> <计数器> <name> HDFS_BYTES_READ </ name> <value> 0 </ value> </ counter> <计数器> <name> HDFS_BYTES_WRITTEN </ name> <value> 0 </ value> </ counter> <计数器> <name> HDFS_READ_OPS </ name> <value> 0 </ value> </ counter> <计数器> <name> HDFS_LARGE_READ_OPS </ name> <value> 0 </ value> </ counter> <计数器> <name> HDFS_WRITE_OPS </ name> <value> 0 </ value> </ counter> </ taskAttemptCounterGroup> <taskAttemptCounterGroup> <counterGroupName> org.apache.hadoop.mapreduce.TaskCounter </ counterGroupName> <计数器> <name> COMBINE_INPUT_RECORDS </ name> <value> 0 </ value> </ counter> <计数器> <name> COMBINE_OUTPUT_RECORDS </ name> <value> 0 </ value> </ counter> <计数器> <name> REDUCE_INPUT_GROUPS </ name> <value> 460 </ value> </ counter> <计数器> <name> REDUCE_SHUFFLE_BYTES </ name> <value> 2235 </ value> </ counter> <计数器> <name> REDUCE_INPUT_RECORDS </ name> <value> 460 </ value> </ counter> <计数器> <name> REDUCE_OUTPUT_RECORDS </ name> <value> 0 </ value> </ counter> <计数器> <name> SPILLED_RECORDS </ name> <value> 0 </ value> </ counter> <计数器> <name> SHUFFLED_MAPS </ name> <value> 1 </ value> </ counter> <计数器> <name> FAILED_SHUFFLE </ name> <value> 0 </ value> </ counter> <计数器> <name> MERGED_MAP_OUTPUTS </ name> <value> 1 </ value> </ counter> <计数器> <name> GC_TIME_MILLIS </ name> <value> 26 </ value> </ counter> <计数器> <名称> CPU_MILLISECONDS </名称> <value> 860 </ value> </ counter> <计数器> <name> PHYSICAL_MEMORY_BYTES </ name> <value> 107839488 </ value> </ counter> <计数器> <name> VIRTUAL_MEMORY_BYTES </ name> <value> 1123147776 </ value> </ counter> <计数器> <name> COMMITTED_HEAP_BYTES </ name> <value> 57475072 </ value> </ counter> </ taskAttemptCounterGroup> <taskAttemptCounterGroup> <counterGroupName>随机播放错误</ counterGroupName> <计数器> <name> BAD_ID </ name> <value> 0 </ value> </ counter> <计数器> <name> CONNECTION </ name> <value> 0 </ value> </ counter> <计数器> <name> IO_ERROR </ name> <value> 0 </ value> </ counter> <计数器> <name> WRONG_LENGTH </ name> <value> 0 </ value> </ counter> <计数器> <name> WRONG_MAP </ name> <value> 0 </ value> </ counter> <计数器> <name> WRONG_REDUCE </ name> <value> 0 </ value> </ counter> </ taskAttemptCounterGroup> <taskAttemptCounterGroup> <counterGroupName> org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter </ counterGroupName> <计数器> <name> BYTES_WRITTEN </ name> <value> 0 </ value> </ counter> </ taskAttemptCounterGroup> </ jobTaskAttemptCounters>