MapReduce应用程序主REST API允许用户获取正在运行的MapReduce应用程序主控的状态。当前,这等效于正在运行的MapReduce作业。该信息包括应用程序主机正在运行的作业以及所有作业详细信息,例如任务,计数器,配置,尝试等。应通过代理访问应用程序主机。可将该代理配置为在资源管理器或单独的主机上运行。代理URL通常如下所示:http:// proxy-http-address:port / proxy / appid。
MapReduce应用程序主信息资源提供有关该Mapreduce应用程序主信息的总体信息。这包括应用程序ID,启动时间,用户,名称等。
当您请求mapreduce应用程序主信息时,该信息将作为信息对象返回。
| 项目 | 数据类型 | 描述 |
|---|---|---|
| appId | 长 | 申请编号 |
| startsOn | 长 | 应用程序启动的时间(从纪元开始以毫秒为单位) |
| 名称 | 串 | 应用程序名称 |
| 用户 | 串 | 启动应用程序的用户的用户名 |
| 经过时间 | 长 | 自应用程序启动以来的时间(以毫秒为单位) |
JSON回应
HTTP请求:
获取http:// proxy-http-address:port / proxy / application_1326232085508_0003 / ws / v1 / mapreduce / info
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{
“信息”:{
“ appId”:“ application_1326232085508_0003”,
“ startedOn”:1326238244047,
“ user”:“ user1”,
“ name”:“睡眠工作”,
“ elapsedTime”:32374
}
}
XML回应
HTTP请求:
接受:application / xml 获取http:// proxy-http-address:port / proxy / application_1326232085508_0003 / ws / v1 / mapreduce / info
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:223 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <信息> <appId> application_1326232085508_0003 </ appId> <name>睡眠作业</ name> <user> user1 </ user> <startedOn> 1326238244047 </ startedOn> <elapsedTime> 32407 </ elapsedTime> </ info>
作业资源提供了在此应用程序主机上运行的作业的列表。另请参见Job API,以获取作业对象的语法。
当您请求作业列表时,该信息将作为作业对象的集合返回。另请参见Job API,以获取作业对象的语法。
| 项目 | 数据类型 | 描述 |
|---|---|---|
| 工作 | 作业对象数组(JSON)/零个或多个作业对象(XML) | 作业对象的集合 |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{
“职位” : {
“工作”:[
{
“ runningReduceAttempts”:1,
“ reduceProgress”:100,
“ failedReduceAttempts”:0,
“ newMapAttempts”:0,
“ mapsRunning”:0,
“ state”:“ RUNNING”,
“ successfulReduceAttempts”:0,
“ reducesRunning”:1,
“ acls”:[
{
“ value”:“”,
“名称”:“ mapreduce.job.acl-modify-job”
},
{
“ value”:“”,
“名称”:“ mapreduce.job.acl-view-job”
}
],
“ reducesPending”:0,
“ user”:“ user1”,
“ reducesTotal”:1
“ mapsCompleted”:1
“ startTime”:1326238769379,
“ id”:“ job_1326232085508_4_4”,
“ successfulMapAttempts”:1
“ runningMapAttempts”:0,
“ newReduceAttempts”:0,
“ name”:“睡眠工作”,
“ mapsPending”:0,
“ elapsedTime”:59377,
“ reducesCompleted”:0,
“ mapProgress”:100,
“诊断”:“”,
“ failedMapAttempts”:0,
“ killedReduceAttempts”:0,
“ mapsTotal”:1
“ uberized”:错误,
“ killedMapAttempts”:0,
“ finishTime”:0
}
]
}
}
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:1214 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?>
<职位>
<工作>
<startTime> 1326238769379 </ startTime>
<finishTime> 0 </ finishTime>
<elapsedTime> 59416 </ elapsedTime>
<id> job_1326232085508_4_4 </ id>
<name>睡眠作业</ name>
<user> user1 </ user>
<状态>运行中</状态>
<mapsTotal> 1 </ mapsTotal>
<mapsCompleted> 1 </ mapsCompleted>
<reducesTotal> 1 </ reducesTotal>
<reducesCompleted> 0 </ reducesCompleted>
<mapProgress> 100.0 </ mapProgress>
<reduceProgress> 100.0 </ reduceProgress>
<mapsPending> 0 </ mapsPending>
<mapsRunning> 0 </ mapsRunning>
<reducesPending> 0 </ reducesPending>
<reducesRunning> 1 </ reducesRunning>
<uberized> false </ uberized>
<diagnostics />
<newReduceAttempts> 0 </ newReduceAttempts>
<runningReduceAttempts> 1 </ runningReduceAttempts>
<failedReduceAttempts> 0 </ failedReduceAttempts>
<killedReduceAttempts> 0 </ killedReduceAttempts>
<successfulReduceAttempts> 0 </ successfulReduceAttempts>
<newMapAttempts> 0 </ newMapAttempts>
<runningMapAttempts> 0 </ runningMapAttempts>
<failedMapAttempts> 0 </ failedMapAttempts>
<killedMapAttempts> 0 </ killedMapAttempts>
<successfulMapAttempts> 1 </ successfulMapAttempts>
<acls>
<name> mapreduce.job.acl-modify-job </ name>
<value> </ value>
</ acls>
<acls>
<name> mapreduce.job.acl-view-job </ name>
<value> </ value>
</ acls>
</ job>
</ jobs>
作业资源包含有关由该应用程序主机启动的特定作业的信息。某些字段仅在用户具有权限时才可访问-取决于acl设置。
| 项目 | 数据类型 | 描述 |
|---|---|---|
| ID | 串 | 工作编号 |
| 名称 | 串 | 工作名称 |
| 用户 | 串 | 用户名 |
| 州 | 串 | 作业状态-有效值为:NEW,INITED,RUNNING,Succeeded,FAILED,KILL_WAIT,KILLED,ERROR |
| 开始时间 | 长 | 作业开始的时间(从纪元开始以毫秒为单位) |
| finishTime | 长 | 作业完成的时间(从开始到现在的毫秒数) |
| 经过时间 | 长 | 自作业开始以来经过的时间(毫秒) |
| mapsTotal | 整型 | 地图总数 |
| mapsCompleted | 整型 | 完成的地图数 |
| reduceTotal | 整型 | 减少总数 |
| reduceCompleted | 整型 | 完成数量减少 |
| 诊断 | 串 | 诊断消息 |
| 超级 | 布尔值 | 指示该作业是否为超级作业-完全在应用程序主服务器中运行 |
| mapsPending | 整型 | 仍要运行的地图数量 |
| mapsRunning | 整型 | 正在运行的地图数 |
| reducePending | 整型 | 减少数量仍需运行 |
| reduceRunning | 整型 | 运行次数减少 |
| newReduce尝试 | 整型 | 新的减少尝试次数 |
| runningReduceAttempts | 整型 | 运行减少尝试的次数 |
| failedReduce尝试 | 整型 | 减少尝试失败的次数 |
| 减少尝试 | 整型 | 减少尝试的失败次数 |
| 成功减少尝试 | 整型 | 成功减少尝试的次数 |
| newMapAttempts | 整型 | 新地图尝试次数 |
| runningMapAttempts | 整型 | 正在运行的地图尝试次数 |
| failedMapAttempts | 整型 | 失败的映射尝试次数 |
| KilledMapAttempts | 整型 | 被杀死的地图尝试次数 |
| successMapAttempts | 整型 | 成功的地图尝试次数 |
| ACL | acls(json)/零个或多个acls对象(xml)的数组 | ACLS对象的集合 |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 服务器:码头(6.1.26) 内容长度:720
响应主体:
{
“工作”:{
“ runningReduceAttempts”:1,
“ reduceProgress”:100,
“ failedReduceAttempts”:0,
“ newMapAttempts”:0,
“ mapsRunning”:0,
“ state”:“ RUNNING”,
“ successfulReduceAttempts”:0,
“ reducesRunning”:1,
“ acls”:[
{
“ value”:“”,
“名称”:“ mapreduce.job.acl-modify-job”
},
{
“ value”:“”,
“名称”:“ mapreduce.job.acl-view-job”
}
],
“ reducesPending”:0,
“ user”:“ user1”,
“ reducesTotal”:1
“ mapsCompleted”:1
“ startTime”:1326238769379,
“ id”:“ job_1326232085508_4_4”,
“ successfulMapAttempts”:1
“ runningMapAttempts”:0,
“ newReduceAttempts”:0,
“ name”:“睡眠工作”,
“ mapsPending”:0,
“ elapsedTime”:59437,
“ reducesCompleted”:0,
“ mapProgress”:100,
“诊断”:“”,
“ failedMapAttempts”:0,
“ killedReduceAttempts”:0,
“ mapsTotal”:1
“ uberized”:错误,
“ killedMapAttempts”:0,
“ finishTime”:0
}
}
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:1201 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?>
<工作>
<startTime> 1326238769379 </ startTime>
<finishTime> 0 </ finishTime>
<elapsedTime> 59474 </ elapsedTime>
<id> job_1326232085508_4_4 </ id>
<name>睡眠作业</ name>
<user> user1 </ user>
<状态>运行中</状态>
<mapsTotal> 1 </ mapsTotal>
<mapsCompleted> 1 </ mapsCompleted>
<reducesTotal> 1 </ reducesTotal>
<reducesCompleted> 0 </ reducesCompleted>
<mapProgress> 100.0 </ mapProgress>
<reduceProgress> 100.0 </ reduceProgress>
<mapsPending> 0 </ mapsPending>
<mapsRunning> 0 </ mapsRunning>
<reducesPending> 0 </ reducesPending>
<reducesRunning> 1 </ reducesRunning>
<uberized> false </ uberized>
<diagnostics />
<newReduceAttempts> 0 </ newReduceAttempts>
<runningReduceAttempts> 1 </ runningReduceAttempts>
<failedReduceAttempts> 0 </ failedReduceAttempts>
<killedReduceAttempts> 0 </ killedReduceAttempts>
<successfulReduceAttempts> 0 </ successfulReduceAttempts>
<newMapAttempts> 0 </ newMapAttempts>
<runningMapAttempts> 0 </ runningMapAttempts>
<failedMapAttempts> 0 </ failedMapAttempts>
<killedMapAttempts> 0 </ killedMapAttempts>
<successfulMapAttempts> 1 </ successfulMapAttempts>
<acls>
<name> mapreduce.job.acl-modify-job </ name>
<value> </ value>
</ acls>
<acls>
<name> mapreduce.job.acl-view-job </ name> <value> </ value>
</ acls>
</ job>
使用作业尝试API,您可以获得代表作业尝试的资源集合。在此资源上运行GET操作时,您将获得一个Job Attempt对象的集合。
当您请求作业尝试列表时,该信息将作为作业尝试对象数组返回。
| 项目 | 数据类型 | 描述 |
|---|---|---|
| 工作尝试 | 作业尝试对象数组(JSON)/零个或多个作业尝试对象(XML) | 作业尝试对象的集合 |
| 项目 | 数据类型 | 描述 |
|---|---|---|
| ID | 串 | 工作尝试编号 |
| nodeId | 串 | 尝试运行的节点的节点ID |
| nodeHttpAddress | 串 | 尝试运行的节点的节点http地址 |
| logsLink | 串 | 作业尝试日志的http链接 |
| containerId | 串 | 尝试作业的容器的ID |
| 开始时间 | 长 | 尝试的开始时间(自纪元以来以毫秒为单位) |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / jobattempts
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{
“ jobAttempts”:{
“ jobAttempt”:[
{
“ nodeId”:“ host.domain.com:8041”,
“ nodeHttpAddress”:“ host.domain.com:8042”,
“ startTime”:1326238773493,
“ id”:1
“ logsLink”:“ http://host.domain.com:8042/node/containerlogs/container_1326232085508_0004_01_000001”,
“ containerId”:“ container_1326232085508_0004_01_000001”
}
]
}
}
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / jobattempts 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:498 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?>
<jobAttempts>
<jobAttempt>
<nodeHttpAddress> host.domain.com:8042 </ nodeHttpAddress>
<nodeId> host.domain.com:8041 </ nodeId>
<id> 1 </ id>
<startTime> 1326238773493 </ startTime>
<containerId> container_1326232085508_0004_01_000001 </ containerId>
<logsLink> http://host.domain.com:8042/node/containerlogs/container_1326232085508_0004_01_000001 </ logsLink>
</ jobAttempt>
</ jobAttempts>
使用作业计数器API,您可以反对代表该作业的所有计数器的资源集合。
| 项目 | 数据类型 | 描述 |
|---|---|---|
| ID | 串 | 工作编号 |
| 柜台集团 | counterGroup对象的数组(JSON)/零个或多个counterGroup对象的XML | 计数器组对象的集合 |
| 项目 | 数据类型 | 描述 |
|---|---|---|
| 名称 | 串 | 柜台名称 |
| reduceCounterValue | 长 | 减少任务的计数器值 |
| mapCounterValue | 长 | 地图任务的对价 |
| totalCounterValue | 长 | 所有任务的对价 |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / counters
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{
“ jobCounters”:{
“ id”:“ job_1326232085508_4_4”,
“ counterGroup”:[
{
“ counterGroupName”:“随机播放错误”,
“柜台”:[
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ BAD_ID”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“连接”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ IO_ERROR”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ WRONG_LENGTH”
},{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ WRONG_MAP”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ WRONG_REDUCE”
}
]
},
{
“ counterGroupName”:“ org.apache.hadoop.mapreduce.FileSystemCounter”,
“柜台”:[
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:2483,
“名称”:“ FILE_BYTES_READ”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:108763,
“名称”:“ FILE_BYTES_WRITTEN”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ FILE_READ_OPS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ FILE_LARGE_READ_OPS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ FILE_WRITE_OPS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:48,
“名称”:“ HDFS_BYTES_READ”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ HDFS_BYTES_WRITTEN”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:1
“名称”:“ HDFS_READ_OPS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ HDFS_LARGE_READ_OPS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ HDFS_WRITE_OPS”
}
]
},
{
“ counterGroupName”:“ org.apache.hadoop.mapreduce.TaskCounter”,
“柜台”:[
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:1
“名称”:“ MAP_INPUT_RECORDS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:1200,
“名称”:“ MAP_OUTPUT_RECORDS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:4800,
“名称”:“ MAP_OUTPUT_BYTES”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:2235,
“名称”:“ MAP_OUTPUT_MATERIALIZED_BYTES”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:48,
“名称”:“ SPLIT_RAW_BYTES”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ COMBINE_INPUT_RECORDS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ COMBINE_OUTPUT_RECORDS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:460,
“名称”:“ REDUCE_INPUT_GROUPS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:2235,
“名称”:“ REDUCE_SHUFFLE_BYTES”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:460,
“名称”:“ REDUCE_INPUT_RECORDS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ REDUCE_OUTPUT_RECORDS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:1200,
“名称”:“ SPILLED_RECORDS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:1
“名称”:“ SHUFFLED_MAPS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ FAILED_SHUFFLE”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:1
“名称”:“ MERGED_MAP_OUTPUTS”
},{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:58
“名称”:“ GC_TIME_MILLIS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:1580,
“名称”:“ CPU_MILLISECONDS”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:462643200,
“名称”:“ PHYSICAL_MEMORY_BYTES”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:2149728256,
“名称”:“ VIRTUAL_MEMORY_BYTES”
},
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:357957632,
“名称”:“ COMMITTED_HEAP_BYTES”
}
]
},
{
“ counterGroupName”:“ org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter”,
“柜台”:[
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ BYTES_READ”
}
]
},
{
“ counterGroupName”:“ org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter”,
“柜台”:[
{
“ reduceCounterValue”:0,
“ mapCounterValue”:0,
“ totalCounterValue”:0,
“名称”:“ BYTES_WRITTEN”
}
]
}
]
}
}
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / counters 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:7027 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?>
<jobCounters>
<id> job_1326232085508_4_4 </ id>
<counterGroup>
<counterGroupName>随机播放错误</ counterGroupName>
<计数器>
<name> BAD_ID </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> CONNECTION </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> IO_ERROR </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> WRONG_LENGTH </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> WRONG_MAP </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> WRONG_REDUCE </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
</ counterGroup>
<counterGroup>
<counterGroupName> org.apache.hadoop.mapreduce.FileSystemCounter </ counterGroupName>
<计数器>
<name> FILE_BYTES_READ </ name>
<totalCounterValue> 2483 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> FILE_BYTES_WRITTEN </ name>
<totalCounterValue> 108763 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<名称> FILE_READ_OPS </名称>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<名称> FILE_LARGE_READ_OPS </名称>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<名称> FILE_WRITE_OPS </名称>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> HDFS_BYTES_READ </ name>
<totalCounterValue> 48 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> HDFS_BYTES_WRITTEN </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> HDFS_READ_OPS </ name>
<totalCounterValue> 1 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> HDFS_LARGE_READ_OPS </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> HDFS_WRITE_OPS </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
</ counterGroup>
<counterGroup>
<counterGroupName> org.apache.hadoop.mapreduce.TaskCounter </ counterGroupName>
<计数器>
<名称> MAP_INPUT_RECORDS </名称>
<totalCounterValue> 1 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<名称> MAP_OUTPUT_RECORDS </名称>
<totalCounterValue> 1200 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<名称> MAP_OUTPUT_BYTES </名称>
<totalCounterValue> 4800 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> MAP_OUTPUT_MATERIALIZED_BYTES </ name>
<totalCounterValue> 2235 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> SPLIT_RAW_BYTES </ name>
<totalCounterValue> 48 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> COMBINE_INPUT_RECORDS </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> COMBINE_OUTPUT_RECORDS </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> REDUCE_INPUT_GROUPS </ name>
<totalCounterValue> 460 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> REDUCE_SHUFFLE_BYTES </ name>
<totalCounterValue> 2235 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> REDUCE_INPUT_RECORDS </ name>
<totalCounterValue> 460 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> REDUCE_OUTPUT_RECORDS </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> SPILLED_RECORDS </ name>
<totalCounterValue> 1200 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> SHUFFLED_MAPS </ name>
<totalCounterValue> 1 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> FAILED_SHUFFLE </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> MERGED_MAP_OUTPUTS </ name>
<totalCounterValue> 1 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> GC_TIME_MILLIS </ name>
<totalCounterValue> 58 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<名称> CPU_MILLISECONDS </名称>
<totalCounterValue> 1580 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> PHYSICAL_MEMORY_BYTES </ name>
<totalCounterValue> 462643200 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> VIRTUAL_MEMORY_BYTES </ name>
<totalCounterValue> 2149728256 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
<计数器>
<name> COMMITTED_HEAP_BYTES </ name>
<totalCounterValue> 357957632 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
</ counterGroup>
<counterGroup>
<counterGroupName> org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter </ counterGroupName>
<计数器>
<name> BYTES_READ </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter> </ counterGroup> <counterGroup>
<counterGroupName> org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter </ counterGroupName>
<counter> <name> BYTES_WRITTEN </ name>
<totalCounterValue> 0 </ totalCounterValue>
<mapCounterValue> 0 </ mapCounterValue>
<reduceCounterValue> 0 </ reduceCounterValue>
</ counter>
</ counterGroup>
</ jobCounters>
作业配置资源包含有关此作业的作业配置的信息。
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / conf
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
如果输出很大,这是输出的一小段。实际输出包含作业配置文件中的每个属性。
{
“ conf”:{
“ path”:“ hdfs://host.domain.com:9000 / user / user1 / .staging / job_1326232085508_0004 / job.xml”,
“财产”:[
{
“ value”:“ / home / hadoop / hdfs / data”,
“ name”:“ dfs.datanode.data.dir”,
“源”:[“ hdfs-site.xml”,“ job.xml”]
},
{
“ value”:“ org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer”,
“ name”:“ hadoop.http.filter.initializers”
“源”:[“以编程方式”,“ job.xml”]
},
{
“ value”:“ / home / hadoop / tmp”,
“名称”:“ mapreduce.cluster.temp.dir”
“源”:[“ mapred-site.xml”]
},
...
]
}
}
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / conf 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:552 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?>
<conf>
<path> hdfs://host.domain.com:9000 / user / user1 / .staging / job_1326232085508_0004 / job.xml </ path>
<属性>
<name> dfs.datanode.data.dir </ name>
<value> / home / hadoop / hdfs / data </ value>
<source> hdfs-site.xml </ source>
<source> job.xml </ source>
</ property>
<属性>
<name> hadoop.http.filter.initializers </ name>
<value> org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer </ value>
<source>以编程方式</ source>
<source> job.xml </ source>
</ property>
<属性>
<name> mapreduce.cluster.temp.dir </ name>
<值> / home / hadoop / tmp </值>
<source> mapred-site.xml </ source>
</ property>
...
</ conf>
使用任务API,您可以获得代表作业的所有任务的资源集合。在此资源上运行GET操作时,您将获得任务对象的集合。
当您请求任务列表时,信息将作为任务对象数组返回。另请参阅任务API,以获取任务对象的语法。
| 项目 | 数据类型 | 描述 |
|---|---|---|
| 任务 | 任务对象数组(JSON)/零个或多个任务对象(XML) | 任务对象的集合 |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{
“任务” : {
“任务”:[
{
“进度”:100,
“ elapsedTime”:2768,
“ state”:“ Succeeded”,
“ startTime”:1326238773493,
“ id”:“ task_1326232085508_4_4_m_0”,
“ type”:“ MAP”,
“ successfulAttempt”:“ attempt_1326232085508_4_4_m_0_0”,
“ finishTime”:1326238776261
},
{
“进度”:100,
“ elapsedTime”:0,
“ state”:“ RUNNING”,
“ startTime”:1326238777460,
“ id”:“ task_1326232085508_4_4_r_0”,
“ type”:“ REDUCE”,
“ successfulAttempt”:“”,
“ finishTime”:0
}
]
}
}
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:603 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?>
<任务>
<任务>
<startTime> 1326238773493 </ startTime>
<finishTime> 1326238776261 </ finishTime>
<elapsedTime> 2768 </ elapsedTime>
<progress> 100.0 </ progress>
<id> task_1326232085508_4_4_m_0 </ id>
<state>成功</ state>
<type> MAP </ type>
<successfulAttempt> attempt_1326232085508_4_4_m_0_0 </ successfulAttempt>
</ task>
<任务>
<startTime> 1326238777460 </ startTime>
<finishTime> 0 </ finishTime>
<elapsedTime> 0 </ elapsedTime>
<progress> 100.0 </ progress>
<id> task_1326232085508_4_4_r_0 </ id>
<状态>运行中</状态>
<type> REDUCE </ type>
<successfulAttempt />
</ task>
</ tasks>
任务资源包含有关作业中特定任务的信息。
| 项目 | 数据类型 | 描述 |
|---|---|---|
| ID | 串 | 任务ID |
| 州 | 串 | 任务的状态-有效值为:NEW,SCHEDULED,RUNNING,Succeeded,FAILED,KILL_WAIT,KILLED |
| 类型 | 串 | 任务类型-MAP或REDUCE |
| 成功尝试 | 串 | 上次成功尝试的ID |
| 进展 | 浮动 | 任务进度百分比 |
| 开始时间 | 长 | 任务开始的时间(从纪元开始以毫秒为单位) |
| finishTime | 长 | 任务完成的时间(从纪元开始以毫秒为单位) |
| 经过时间 | 长 | 自应用程序启动以来经过的时间(以毫秒为单位) |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{
“任务”:{
“进度”:100,
“ elapsedTime”:0,
“ state”:“ RUNNING”,
“ startTime”:1326238777460,
“ id”:“ task_1326232085508_4_4_r_0”,
“ type”:“ REDUCE”,
“ successfulAttempt”:“”,
“ finishTime”:0
}
}
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:299 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <任务> <startTime> 1326238777460 </ startTime> <finishTime> 0 </ finishTime> <elapsedTime> 0 </ elapsedTime> <progress> 100.0 </ progress> <id> task_1326232085508_4_4_r_0 </ id> <状态>运行中</状态> <type> REDUCE </ type> <successfulAttempt /> </ task>
使用任务计数器API,您可以反对代表该任务所有计数器的资源集合。
| 项目 | 数据类型 | 描述 |
|---|---|---|
| ID | 串 | 任务ID |
| taskcounterGroup | counterGroup对象的数组(JSON)/零个或多个counterGroup对象的XML | 计数器组对象的集合 |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 / counters
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{
“ jobTaskCounters”:{
“ id”:“ task_1326232085508_4_4_r_0”,
“ taskCounterGroup”:[
{
“ counterGroupName”:“ org.apache.hadoop.mapreduce.FileSystemCounter”,
“柜台”:[
{
“值”:2363,
“名称”:“ FILE_BYTES_READ”
},
{
“值”:54372,
“名称”:“ FILE_BYTES_WRITTEN”
},
{
“值”:0,
“名称”:“ FILE_READ_OPS”
},
{
“值”:0,
“名称”:“ FILE_LARGE_READ_OPS”
},
{
“值”:0,
“名称”:“ FILE_WRITE_OPS”
},
{
“值”:0,
“名称”:“ HDFS_BYTES_READ”
},
{
“值”:0,
“名称”:“ HDFS_BYTES_WRITTEN”
},
{
“值”:0,
“名称”:“ HDFS_READ_OPS”
},
{
“值”:0,
“名称”:“ HDFS_LARGE_READ_OPS”
},
{
“值”:0,
“名称”:“ HDFS_WRITE_OPS”
}
]
},
{
“ counterGroupName”:“ org.apache.hadoop.mapreduce.TaskCounter”,
“柜台”:[
{
“值”:0,
“名称”:“ COMBINE_INPUT_RECORDS”
},
{
“值”:0,
“名称”:“ COMBINE_OUTPUT_RECORDS”
},
{
“值”:460,
“名称”:“ REDUCE_INPUT_GROUPS”
},
{
“值”:2235,
“名称”:“ REDUCE_SHUFFLE_BYTES”
},
{
“值”:460,
“名称”:“ REDUCE_INPUT_RECORDS”
},
{
“值”:0,
“名称”:“ REDUCE_OUTPUT_RECORDS”
},
{
“值”:0,
“名称”:“ SPILLED_RECORDS”
},
{
“值”:1
“名称”:“ SHUFFLED_MAPS”
},
{
“值”:0,
“名称”:“ FAILED_SHUFFLE”
},
{
“值”:1
“名称”:“ MERGED_MAP_OUTPUTS”
},
{
“值”:26,
“名称”:“ GC_TIME_MILLIS”
},
{
“值”:860,
“名称”:“ CPU_MILLISECONDS”
},
{
“值”:107839488,
“名称”:“ PHYSICAL_MEMORY_BYTES”
},
{
“值”:1123147776,
“名称”:“ VIRTUAL_MEMORY_BYTES”
},
{
“值”:57475072,
“名称”:“ COMMITTED_HEAP_BYTES”
}
]
},
{
“ counterGroupName”:“随机播放错误”,
“柜台”:[
{
“值”:0,
“名称”:“ BAD_ID”
},
{
“值”:0,
“名称”:“连接”
},
{
“值”:0,
“名称”:“ IO_ERROR”
},
{
“值”:0,
“名称”:“ WRONG_LENGTH”
},
{
“值”:0,
“名称”:“ WRONG_MAP”
},
{
“值”:0,
“名称”:“ WRONG_REDUCE”
}
]
},
{
“ counterGroupName”:“ org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter”,
“柜台”:[
{
“值”:0,
“名称”:“ BYTES_WRITTEN”
}
]
}
]
}
}
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 / counters 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:2660 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?>
<jobTaskCounters>
<id> task_1326232085508_4_4_r_0 </ id>
<taskCounterGroup>
<counterGroupName> org.apache.hadoop.mapreduce.FileSystemCounter </ counterGroupName>
<计数器>
<name> FILE_BYTES_READ </ name>
<value> 2363 </ value>
</ counter>
<计数器>
<name> FILE_BYTES_WRITTEN </ name>
<value> 54372 </ value>
</ counter>
<计数器>
<名称> FILE_READ_OPS </名称>
<value> 0 </ value>
</ counter>
<计数器>
<名称> FILE_LARGE_READ_OPS </名称>
<value> 0 </ value>
</ counter>
<计数器>
<名称> FILE_WRITE_OPS </名称>
<value> 0 </ value>
</ counter>
<计数器>
<name> HDFS_BYTES_READ </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> HDFS_BYTES_WRITTEN </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> HDFS_READ_OPS </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> HDFS_LARGE_READ_OPS </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> HDFS_WRITE_OPS </ name>
<value> 0 </ value>
</ counter>
</ taskCounterGroup>
<taskCounterGroup>
<counterGroupName> org.apache.hadoop.mapreduce.TaskCounter </ counterGroupName>
<计数器>
<name> COMBINE_INPUT_RECORDS </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> COMBINE_OUTPUT_RECORDS </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> REDUCE_INPUT_GROUPS </ name>
<value> 460 </ value>
</ counter>
<计数器>
<name> REDUCE_SHUFFLE_BYTES </ name>
<value> 2235 </ value>
</ counter>
<计数器>
<name> REDUCE_INPUT_RECORDS </ name>
<value> 460 </ value>
</ counter>
<计数器>
<name> REDUCE_OUTPUT_RECORDS </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> SPILLED_RECORDS </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> SHUFFLED_MAPS </ name>
<value> 1 </ value>
</ counter>
<计数器>
<name> FAILED_SHUFFLE </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> MERGED_MAP_OUTPUTS </ name>
<value> 1 </ value>
</ counter>
<计数器>
<name> GC_TIME_MILLIS </ name>
<value> 26 </ value>
</ counter>
<计数器>
<名称> CPU_MILLISECONDS </名称>
<value> 860 </ value>
</ counter>
<计数器>
<name> PHYSICAL_MEMORY_BYTES </ name>
<value> 107839488 </ value>
</ counter>
<计数器>
<name> VIRTUAL_MEMORY_BYTES </ name>
<value> 1123147776 </ value>
</ counter>
<计数器>
<name> COMMITTED_HEAP_BYTES </ name>
<value> 57475072 </ value>
</ counter>
</ taskCounterGroup>
<taskCounterGroup>
<counterGroupName>随机播放错误</ counterGroupName>
<计数器>
<name> BAD_ID </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> CONNECTION </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> IO_ERROR </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> WRONG_LENGTH </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> WRONG_MAP </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> WRONG_REDUCE </ name>
<value> 0 </ value>
</ counter>
</ taskCounterGroup>
<taskCounterGroup>
<counterGroupName> org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter </ counterGroupName>
<计数器>
<name> BYTES_WRITTEN </ name>
<value> 0 </ value>
</ counter>
</ taskCounterGroup>
</ jobTaskCounters>
使用任务尝试API,您可以获得代表作业中任务尝试的资源集合。在此资源上运行GET操作时,将获得“任务尝试对象”的集合。
当您请求任务尝试列表时,该信息将作为任务尝试对象数组返回。另请参阅Task Attempt API,以获取任务对象的语法。
| 项目 | 数据类型 | 描述 |
|---|---|---|
| taskAttempt | 任务尝试对象(JSON)/零个或多个任务尝试对象(XML)的数组 | 任务尝试对象的集合 |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 /尝试
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{
“ taskAttempts”:{
“ taskAttempt”:[
{
“ elapsedMergeTime”:47,
“ shuffleFinishTime”:1326238780052,
“ assignedContainerId”:“ container_1326232085508_0004_01_000003”,
“进度”:100,
“ elapsedTime”:0,
“ state”:“ RUNNING”,
“ elapsedShuffleTime”:2592,
“ mergeFinishTime”:1326238780099,
“ rack”:“ / 98.139.92.0”,
“ elapsedReduceTime”:0,
“ nodeHttpAddress”:“ host.domain.com:8042”,
“ type”:“ REDUCE”,
“ startTime”:1326238777460,
“ id”:“ attempt_1326232085508_4_4_r_0_0”,
“ finishTime”:0
}
]
}
}
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 /尝试 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:807 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?>
<taskAttempts>
<taskAttempt>
<startTime> 1326238777460 </ startTime>
<finishTime> 0 </ finishTime>
<elapsedTime> 0 </ elapsedTime>
<progress> 100.0 </ progress>
<id> attempt_1326232085508_4_4_r_0_0 </ id>
<rack> /98.139.92.0 </ rack>
<状态>运行中</状态>
<nodeHttpAddress> host.domain.com:8042 </ nodeHttpAddress>
<type> REDUCE </ type>
<assignedContainerId> container_1326232085508_0004_01_000003 </ assignedContainerId>
<shuffleFinishTime> 1326238780052 </ shuffleFinishTime>
<mergeFinishTime> 1326238780099 </ mergeFinishTime>
<elapsedShuffleTime> 2592 </ elapsedShuffleTime>
<elapsedMergeTime> 47 </ elapsedMergeTime>
<elapsedReduceTime> 0 </ elapsedReduceTime>
</ taskAttempt>
</ taskAttempts>
任务尝试资源包含有关作业中特定任务尝试的信息。
| 项目 | 数据类型 | 描述 |
|---|---|---|
| ID | 串 | 任务ID |
| 架 | 串 | 机架 |
| 州 | 串 | 任务尝试的状态-有效值为:NEW,UNASSIGNED,ASSIGNED,RUNNING,COMMIT_PENDING,SUCCESS_CONTAINER_CLEANUP,SUCCEEDED,FAIL_CONTAINER_CLEANUP,FAIL_TASK_CLEANUP,FAILED,KILL_CONTAINER_CLEANUP,KILL_TAIL_ |
| 类型 | 串 | 任务类型 |
| AssignedContainerId | 串 | 此尝试分配给的容器ID |
| nodeHttpAddress | 串 | 尝试执行此任务的节点的http地址 |
| 诊断 | 串 | 诊断消息 |
| 进展 | 浮动 | 任务尝试的进度百分比 |
| 开始时间 | 长 | 任务尝试开始的时间(自时期起以毫秒为单位) |
| finishTime | 长 | 任务尝试完成的时间(自纪元以来以毫秒为单位) |
| 经过时间 | 长 | 自任务开始尝试以来经过的时间(以毫秒为单位) |
对于减少任务尝试,您还具有以下字段:
| 项目 | 数据类型 | 描述 |
|---|---|---|
| shuffleFinishTime | 长 | 随机播放结束的时间(自纪元以来以毫秒为单位) |
| mergeFinishTime | 长 | 合并完成的时间(自纪元以来以毫秒为单位) |
| 经过的ShuffleTime | 长 | 随机播放阶段完成所需的时间(减少任务开始和随机播放完成之间的时间,以毫秒为单位) |
| 经过的合并时间 | 长 | 合并阶段完成所需的时间(混洗完成和合并完成之间的时间,以毫秒为单位) |
| 经过减少时间 | 长 | 还原阶段完成所需的时间(从合并完成到还原任务结束之间的时间,以毫秒为单位) |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 / attempts / attempt_1326232085508_4_4_r_0_0
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{
“ taskAttempt”:{
“ elapsedMergeTime”:47,
“ shuffleFinishTime”:1326238780052,
“ assignedContainerId”:“ container_1326232085508_0004_01_000003”,
“进度”:100,
“ elapsedTime”:0,
“ state”:“ RUNNING”,
“ elapsedShuffleTime”:2592,
“ mergeFinishTime”:1326238780099,
“ rack”:“ / 98.139.92.0”,
“ elapsedReduceTime”:0,
“ nodeHttpAddress”:“ host.domain.com:8042”,
“ startTime”:1326238777460,
“ id”:“ attempt_1326232085508_4_4_r_0_0”,
“ type”:“ REDUCE”,
“ finishTime”:0
}
}
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 / attempts / attempt_1326232085508_4_4_r_0_0 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:691 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <taskAttempt> <startTime> 1326238777460 </ startTime> <finishTime> 0 </ finishTime> <elapsedTime> 0 </ elapsedTime> <progress> 100.0 </ progress> <id> attempt_1326232085508_4_4_r_0_0 </ id> <rack> /98.139.92.0 </ rack> <状态>运行中</状态> <nodeHttpAddress> host.domain.com:8042 </ nodeHttpAddress> <type> REDUCE </ type> <assignedContainerId> container_1326232085508_0004_01_000003 </ assignedContainerId> <shuffleFinishTime> 1326238780052 </ shuffleFinishTime> <mergeFinishTime> 1326238780099 </ mergeFinishTime> <elapsedShuffleTime> 2592 </ elapsedShuffleTime> <elapsedMergeTime> 47 </ elapsedMergeTime> <elapsedReduceTime> 0 </ elapsedReduceTime> </ taskAttempt>
使用任务尝试状态API,您可以使用状态设置为“ KILLED”的PUT请求修改运行任务尝试的状态,从而查询提交的任务尝试的状态以及终止正在运行的任务尝试。要执行PUT操作,必须为AM Web服务设置身份验证。另外,您必须被授权杀死任务尝试。当前,您只能将状态更改为“已杀死”。尝试将状态更改为其他任何状态都会导致400错误响应。下面是未授权和错误请求错误的示例。当您执行成功的PUT时,初始响应可能是202。您可以通过重复PUT请求直到得到200,使用GET方法查询状态或查询任务尝试信息并检查,来确认该应用已被杀死。状态。在以下示例中,
请注意,为了终止任务尝试,必须为HTTP接口设置身份验证筛选器。该功能要求在HttpServletRequest中设置用户名。如果未设置任何过滤器,则响应将为“未经授权”响应。
此功能目前处于Alpha阶段,将来可能会更改。
当您请求应用程序状态时,返回的信息包含以下字段
| 项目 | 数据类型 | 描述 |
|---|---|---|
| 州 | 串 | 应用程序状态-可以是“ NEW”,“ STARTING”,“ RUNNING”,“ COMMIT_PENDING”,“ Succeeded”,“ FAILED”,“ KILLED”之一 |
JSON回应
HTTP请求
GET http:// proxy-http-address:port / proxy / application_1429692837321_0001 / ws / v1 / mapreduce / jobs / job_1429692837321_0001 / tasks / task_1429692837321_0001_m_000000 / attempts / attempt_1429692837321_0001_m_000000_0 / state
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 服务器:码头(6.1.26) 内容长度:20
响应主体:
{
“ state”:“ STARTING”
}
HTTP请求
PUT http:// proxy-http-address:port / proxy / application_1429692837321_0001 / ws / v1 / mapreduce / jobs / job_1429692837321_0001 / tasks / task_1429692837321_0001_m_000000 / attempts / attempt_1429692837321_0001_m_000000_0 / state
请求正文:
{
“ state”:“ KILLED”
}
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 服务器:码头(6.1.26) 内容长度:18
响应主体:
{
“ state”:“ KILLED”
}
XML回应
HTTP请求
GET http:// proxy-http-address:port / proxy / application_1429692837321_0001 / ws / v1 / mapreduce / jobs / job_1429692837321_0001 / tasks / task_1429692837321_0001_m_000000 / attempts / attempt_1429692837321_0001_m_000000_0 / state
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 服务器:码头(6.1.26) 内容长度:121
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <jobTaskAttemptState> <状态>开始</状态> </ jobTaskAttemptState>
HTTP请求
PUT http:// proxy-http-address:port / proxy / application_1429692837321_0001 / ws / v1 / mapreduce / jobs / job_1429692837321_0001 / tasks / task_1429692837321_0001_m_000000 / attempts / attempt_1429692837321_0001_m_000000_0 / state
请求正文:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <jobTaskAttemptState> <state> KILLED </ state> </ jobTaskAttemptState>
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 服务器:码头(6.1.26) 内容长度:121
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <jobTaskAttemptState> <state> KILLED </ state> </ jobTaskAttemptState>
未经授权的错误响应
HTTP请求
PUT http:// proxy-http-address:port / proxy / application_1429692837321_0001 / ws / v1 / mapreduce / jobs / job_1429692837321_0001 / tasks / task_1429692837321_0001_m_000000 / attempts / attempt_1429692837321_0001_m_000000_0 / state
请求正文:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <jobTaskAttemptState> <state> KILLED </ state> </ jobTaskAttemptState>
响应标题:
HTTP / 1.1 403未经授权 内容类型:application / json 服务器:码头(6.1.26)
错误的请求错误响应
HTTP请求
PUT http:// proxy-http-address:port / proxy / application_1429692837321_0001 / ws / v1 / mapreduce / jobs / job_1429692837321_0001 / tasks / task_1429692837321_0001_m_000000 / attempts / attempt_1429692837321_0001_m_000000_0 / state
请求正文:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <jobTaskAttemptState> <状态>运行中</状态> </ jobTaskAttemptState>
响应标题:
HTTP / 1.1 400 内容长度:295 内容类型:application / xml 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?> <RemoteException> <exception> BadRequestException </ exception> <message> java.lang.Exception:仅允许将“ KILLED”作为目标状态。</ message> <javaClassName> org.apache.hadoop.yarn.webapp.BadRequestException </ javaClassName> </ RemoteException>
使用任务尝试计数器API,您可以反对代表该任务尝试的所有计数器的资源集合。
| 项目 | 数据类型 | 描述 |
|---|---|---|
| ID | 串 | 任务尝试ID |
| taskAttemptcounterGroup | 任务尝试counterGroup对象(JSON)/零个或多个任务尝试counterGroup对象(XML)的数组 | 任务尝试计数器组对象的集合 |
| 项目 | 数据类型 | 描述 |
|---|---|---|
| counterGroupName | 串 | 柜台组名称 |
| 计数器 | 计数器对象数组(JSON)/零个或多个计数器对象(XML) | 柜台对象的集合 |
JSON回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 / attempts / attempt_1326232085508_4_4_r_0_0 / counters
响应标题:
HTTP / 1.1 200 OK 内容类型:application / json 传输编码:分块 服务器:码头(6.1.26)
响应主体:
{
“ jobTaskAttemptCounters”:{
“ taskAttemptCounterGroup”:[
{
“ counterGroupName”:“ org.apache.hadoop.mapreduce.FileSystemCounter”,
“柜台”:[
{
“值”:2363,
“名称”:“ FILE_BYTES_READ”
},
{
“值”:54372,
“名称”:“ FILE_BYTES_WRITTEN”
},
{
“值”:0,
“名称”:“ FILE_READ_OPS”
},
{
“值”:0,
“名称”:“ FILE_LARGE_READ_OPS”
},
{
“值”:0,
“名称”:“ FILE_WRITE_OPS”
},
{
“值”:0,
“名称”:“ HDFS_BYTES_READ”
},
{
“值”:0,
“名称”:“ HDFS_BYTES_WRITTEN”
},
{
“值”:0,
“名称”:“ HDFS_READ_OPS”
},
{
“值”:0,
“名称”:“ HDFS_LARGE_READ_OPS”
},
{
“值”:0,
“名称”:“ HDFS_WRITE_OPS”
}
]
},
{
“ counterGroupName”:“ org.apache.hadoop.mapreduce.TaskCounter”,
“柜台”:[
{
“值”:0,
“名称”:“ COMBINE_INPUT_RECORDS”
},
{
“值”:0,
“名称”:“ COMBINE_OUTPUT_RECORDS”
},
{
“值”:460,
“名称”:“ REDUCE_INPUT_GROUPS”
},
{
“值”:2235,
“名称”:“ REDUCE_SHUFFLE_BYTES”
},
{
“值”:460,
“名称”:“ REDUCE_INPUT_RECORDS”
},
{
“值”:0,
“名称”:“ REDUCE_OUTPUT_RECORDS”
},
{
“值”:0,
“名称”:“ SPILLED_RECORDS”
},
{
“值”:1
“名称”:“ SHUFFLED_MAPS”
},
{
“值”:0,
“名称”:“ FAILED_SHUFFLE”
},
{
“值”:1
“名称”:“ MERGED_MAP_OUTPUTS”
},
{
“值”:26,
“名称”:“ GC_TIME_MILLIS”
},
{
“值”:860,
“名称”:“ CPU_MILLISECONDS”
},
{
“值”:107839488,
“名称”:“ PHYSICAL_MEMORY_BYTES”
},
{
“值”:1123147776,
“名称”:“ VIRTUAL_MEMORY_BYTES”
},
{
“值”:57475072,
“名称”:“ COMMITTED_HEAP_BYTES”
}
]
},
{
“ counterGroupName”:“随机播放错误”,
“柜台”:[
{
“值”:0,
“名称”:“ BAD_ID”
},
{
“值”:0,
“名称”:“连接”
},
{
“值”:0,
“名称”:“ IO_ERROR”
},
{
“值”:0,
“名称”:“ WRONG_LENGTH”
},
{
“值”:0,
“名称”:“ WRONG_MAP”
},
{
“值”:0,
“名称”:“ WRONG_REDUCE”
}
]
},
{
“ counterGroupName”:“ org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter”,
“柜台”:[
{
“值”:0,
“名称”:“ BYTES_WRITTEN”
}
]
}
],
“ id”:“ attempt_1326232085508_4_4_r_0_0”
}
}
XML回应
HTTP请求:
GET http:// proxy-http-address:port / proxy / application_1326232085508_0004 / ws / v1 / mapreduce / jobs / job_1326232085508_4_4 / tasks / task_1326232085508_4_4_r_0 / attempts / attempt_1326232085508_4_4_r_0_0 / counters 接受:application / xml
响应标题:
HTTP / 1.1 200 OK 内容类型:application / xml 内容长度:2735 服务器:码头(6.1.26)
响应主体:
<?xml版本=“ 1.0”编码=“ UTF-8”独立=“是”?>
<jobTaskAttemptCounters>
<id> attempt_1326232085508_4_4_r_0_0 </ id>
<taskAttemptCounterGroup>
<counterGroupName> org.apache.hadoop.mapreduce.FileSystemCounter </ counterGroupName>
<计数器>
<name> FILE_BYTES_READ </ name>
<value> 2363 </ value>
</ counter>
<计数器>
<name> FILE_BYTES_WRITTEN </ name>
<value> 54372 </ value>
</ counter>
<计数器>
<名称> FILE_READ_OPS </名称>
<value> 0 </ value>
</ counter>
<计数器>
<名称> FILE_LARGE_READ_OPS </名称>
<value> 0 </ value>
</ counter>
<计数器>
<名称> FILE_WRITE_OPS </名称>
<value> 0 </ value>
</ counter>
<计数器>
<name> HDFS_BYTES_READ </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> HDFS_BYTES_WRITTEN </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> HDFS_READ_OPS </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> HDFS_LARGE_READ_OPS </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> HDFS_WRITE_OPS </ name>
<value> 0 </ value>
</ counter>
</ taskAttemptCounterGroup>
<taskAttemptCounterGroup>
<counterGroupName> org.apache.hadoop.mapreduce.TaskCounter </ counterGroupName>
<计数器>
<name> COMBINE_INPUT_RECORDS </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> COMBINE_OUTPUT_RECORDS </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> REDUCE_INPUT_GROUPS </ name>
<value> 460 </ value>
</ counter>
<计数器>
<name> REDUCE_SHUFFLE_BYTES </ name>
<value> 2235 </ value>
</ counter>
<计数器>
<name> REDUCE_INPUT_RECORDS </ name>
<value> 460 </ value>
</ counter>
<计数器>
<name> REDUCE_OUTPUT_RECORDS </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> SPILLED_RECORDS </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> SHUFFLED_MAPS </ name>
<value> 1 </ value>
</ counter>
<计数器>
<name> FAILED_SHUFFLE </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> MERGED_MAP_OUTPUTS </ name>
<value> 1 </ value>
</ counter>
<计数器>
<name> GC_TIME_MILLIS </ name>
<value> 26 </ value>
</ counter>
<计数器>
<名称> CPU_MILLISECONDS </名称>
<value> 860 </ value>
</ counter>
<计数器>
<name> PHYSICAL_MEMORY_BYTES </ name>
<value> 107839488 </ value>
</ counter>
<计数器>
<name> VIRTUAL_MEMORY_BYTES </ name>
<value> 1123147776 </ value>
</ counter>
<计数器>
<name> COMMITTED_HEAP_BYTES </ name>
<value> 57475072 </ value>
</ counter>
</ taskAttemptCounterGroup>
<taskAttemptCounterGroup>
<counterGroupName>随机播放错误</ counterGroupName>
<计数器>
<name> BAD_ID </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> CONNECTION </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> IO_ERROR </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> WRONG_LENGTH </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> WRONG_MAP </ name>
<value> 0 </ value>
</ counter>
<计数器>
<name> WRONG_REDUCE </ name>
<value> 0 </ value>
</ counter>
</ taskAttemptCounterGroup>
<taskAttemptCounterGroup>
<counterGroupName> org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter </ counterGroupName>
<计数器>
<name> BYTES_WRITTEN </ name>
<value> 0 </ value>
</ counter>
</ taskAttemptCounterGroup>
</ jobTaskAttemptCounters>